[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:56:03 dist-test-slave-gp6n krb5kdc[22452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:56:03 dist-test-slave-gp6n krb5kdc[22452](info): set up 2 sockets
Aug 09 12:56:03 dist-test-slave-gp6n krb5kdc[22452](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 12:56:04.991950 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.001s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:56:05 dist-test-slave-gp6n krb5kdc[22452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:56:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:56:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 12:56:05.196225 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:45659
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:33403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:45659
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:05.588999 22465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:05.589409 22465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:05.589519 22465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:05.596764 22465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:56:05.596874 22465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:05.596956 22465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:05.597018 22465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:56:05.597062 22465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:56:05.607571 22465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:45659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:45659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:05.609381 22465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:05.611234 22465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:05.633965 22471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:05.634855 22473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:05.635064 22470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:05.638809 22465 server_base.cc:1034] running on GCE node
I20260809 12:56:05.639917 22465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:05.642174 22465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:05.643435 22465 hybrid_clock.cc:648] HybridClock initialized: now 1786280165643332 us; error 97 us; skew 500 ppm
I20260809 12:56:05.643971 22465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:05 dist-test-slave-gp6n krb5kdc[22452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280165, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:05.647644 22465 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:56:05.650203 22465 webserver.cc:460] Webserver started at http://127.21.235.126:34525/ using document root <none> and password file <none>
I20260809 12:56:05.651333 22465 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:05.651445 22465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:05.651883 22465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:05.656219 22465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "2a663ce2b5e544f9a89bac5b75a0ac68"
format_stamp: "Formatted at 2026-08-09 12:56:05 on dist-test-slave-gp6n"
server_key: "b05b5e8a15070df555de0ee6262ebdf9"
server_key_iv: "09557e98537d864de7c750fddbdd480f"
server_key_version: "encryptionkey@0"
I20260809 12:56:05.657063 22465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "2a663ce2b5e544f9a89bac5b75a0ac68"
format_stamp: "Formatted at 2026-08-09 12:56:05 on dist-test-slave-gp6n"
server_key: "b05b5e8a15070df555de0ee6262ebdf9"
server_key_iv: "09557e98537d864de7c750fddbdd480f"
server_key_version: "encryptionkey@0"
I20260809 12:56:05.664842 22465 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260809 12:56:05.670197 22482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:05.672273 22465 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260809 12:56:05.672559 22465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "2a663ce2b5e544f9a89bac5b75a0ac68"
format_stamp: "Formatted at 2026-08-09 12:56:05 on dist-test-slave-gp6n"
server_key: "b05b5e8a15070df555de0ee6262ebdf9"
server_key_iv: "09557e98537d864de7c750fddbdd480f"
server_key_version: "encryptionkey@0"
I20260809 12:56:05.672775 22465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:05.715866 22465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:05.721006 22465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:05.721427 22465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:05.745245 22465 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:45659
I20260809 12:56:05.745329 22539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:45659 every 8 connection(s)
I20260809 12:56:05.747663 22465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:56:05.754824 22540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:05.757076 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 22465
I20260809 12:56:05.757570 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:56:05.761118 22445 external_mini_cluster.cc:1428] Setting key 9a7174a03f2d27df7ff424cc0c0497d3
I20260809 12:56:05.776106 22540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68: Bootstrap starting.
I20260809 12:56:05.782642 22540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:05.785154 22540 log.cc:824] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:05.789691 22540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68: No bootstrap required, opened a new log
Aug 09 12:56:05 dist-test-slave-gp6n krb5kdc[22452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:05.801627 22540 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a663ce2b5e544f9a89bac5b75a0ac68" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 45659 } }
I20260809 12:56:05.802201 22540 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:05.802452 22540 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a663ce2b5e544f9a89bac5b75a0ac68, State: Initialized, Role: FOLLOWER
I20260809 12:56:05.803989 22540 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a663ce2b5e544f9a89bac5b75a0ac68" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 45659 } }
I20260809 12:56:05.804477 22540 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:56:05.804836 22540 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:56:05.805155 22540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:05.809250 22540 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a663ce2b5e544f9a89bac5b75a0ac68" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 45659 } }
I20260809 12:56:05.810673 22540 leader_election.cc:302] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a663ce2b5e544f9a89bac5b75a0ac68; no voters:
I20260809 12:56:05.811841 22540 leader_election.cc:288] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:56:05.812047 22546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:05.815255 22540 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:56:05.816098 22542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:05.768880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55468 (local address 127.21.235.126:45659)
0809 12:56:05.771317 (+ 2437us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:05.771335 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:05.773444 (+ 2109us) server_negotiation.cc:407] Connection header received
0809 12:56:05.785046 (+ 11602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:05.785096 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:05.785954 (+ 858us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:05.786597 (+ 643us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:05.789846 (+ 3249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:05.792990 (+ 3144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:05.794897 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:05.796231 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:05.796635 (+ 404us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:05.802925 (+ 6290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:05.802998 (+ 73us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:05.803043 (+ 45us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:05.803118 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:05.808222 (+ 5104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:05.809079 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:05.809091 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:05.809101 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:05.809444 (+ 343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:05.809958 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:05.809963 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:05.809967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:05.811199 (+ 1232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:05.811772 (+ 573us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:05.813025 (+ 1253us) server_negotiation.cc:299] Negotiation successful
0809 12:56:05.813629 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1980,"thread_start_us":1596,"threads_started":1}
I20260809 12:56:05.819972 22546 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68 [term 1 LEADER]: Becoming Leader. State: Replica: 2a663ce2b5e544f9a89bac5b75a0ac68, State: Running, Role: LEADER
I20260809 12:56:05.822017 22546 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a663ce2b5e544f9a89bac5b75a0ac68" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 45659 } }
I20260809 12:56:05.827975 22547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a663ce2b5e544f9a89bac5b75a0ac68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a663ce2b5e544f9a89bac5b75a0ac68" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 45659 } } }
I20260809 12:56:05.828490 22547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68 [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:05.828526 22548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a663ce2b5e544f9a89bac5b75a0ac68. Latest consensus state: current_term: 1 leader_uuid: "2a663ce2b5e544f9a89bac5b75a0ac68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a663ce2b5e544f9a89bac5b75a0ac68" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 45659 } } }
I20260809 12:56:05.828923 22548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68 [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:05.832485 22566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:56:05.841032 22566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:56:05.854835 22566 catalog_manager.cc:1347] Generated new cluster ID: fc82b4c9ed824ababffd07719a606c0c
I20260809 12:56:05.854967 22566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:56:05.866757 22566 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:56:05.868299 22566 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:56:05.882835 22566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a663ce2b5e544f9a89bac5b75a0ac68: Generated new TSK 0
I20260809 12:56:05.883879 22566 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
I20260809 12:56:05.938931 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:45659
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:33403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:06.328326 22573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:06.328703 22573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:06.328858 22573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:06.336550 22573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:06.336658 22573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:06.336820 22573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:56:06.347608 22573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:45659
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:06.348959 22573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:06.350387 22573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:06.367841 22580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:06.367789 22582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:06.367789 22584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:06.369433 22573 server_base.cc:1034] running on GCE node
I20260809 12:56:06.370198 22573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:06.371376 22573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:06.372633 22573 hybrid_clock.cc:648] HybridClock initialized: now 1786280166372566 us; error 53 us; skew 500 ppm
I20260809 12:56:06.373092 22573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:06 dist-test-slave-gp6n krb5kdc[22452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280166, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:06.381256 22573 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 12:56:06.383208 22573 webserver.cc:460] Webserver started at http://127.21.235.65:41485/ using document root <none> and password file <none>
I20260809 12:56:06.383893 22573 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:06.384006 22573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:06.384377 22573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:06.387850 22573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "b2badc62acc6425aa342029ccb3f5dc3"
format_stamp: "Formatted at 2026-08-09 12:56:06 on dist-test-slave-gp6n"
server_key: "2514bab15a8eaa16f638bef97cd7ae99"
server_key_iv: "906395ac2162318a315eac82f85acd4b"
server_key_version: "encryptionkey@0"
I20260809 12:56:06.388664 22573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "b2badc62acc6425aa342029ccb3f5dc3"
format_stamp: "Formatted at 2026-08-09 12:56:06 on dist-test-slave-gp6n"
server_key: "2514bab15a8eaa16f638bef97cd7ae99"
server_key_iv: "906395ac2162318a315eac82f85acd4b"
server_key_version: "encryptionkey@0"
I20260809 12:56:06.394256 22573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:56:06.398167 22592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:06.399627 22573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:56:06.399801 22573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "b2badc62acc6425aa342029ccb3f5dc3"
format_stamp: "Formatted at 2026-08-09 12:56:06 on dist-test-slave-gp6n"
server_key: "2514bab15a8eaa16f638bef97cd7ae99"
server_key_iv: "906395ac2162318a315eac82f85acd4b"
server_key_version: "encryptionkey@0"
I20260809 12:56:06.400005 22573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:06.415511 22573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:06.420390 22573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:06.420740 22573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:06.422075 22573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:06.424271 22573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:06.424382 22573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:06.424584 22573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:06.424671 22573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:06.468874 22573 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:37665
I20260809 12:56:06.468932 22720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:37665 every 8 connection(s)
I20260809 12:56:06.470502 22573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:56:06.479861 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 22573
I20260809 12:56:06.480103 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:56:06.480984 22445 external_mini_cluster.cc:1428] Setting key 0f3e909b70a4803cdc1294d356fd84b3
Aug 09 12:56:06 dist-test-slave-gp6n krb5kdc[22452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280166, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:06.497777 22723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:06.473990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:45107 (local address 127.21.235.126:45659)
0809 12:56:06.474560 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:06.474567 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:06.475466 (+ 899us) server_negotiation.cc:407] Connection header received
0809 12:56:06.478002 (+ 2536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:06.478008 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:06.478113 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:06.478219 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:06.480441 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:06.481710 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:06.484373 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:06.485323 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:06.485489 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:06.489656 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:06.489665 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:06.489670 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:06.489718 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:06.493970 (+ 4252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:06.494860 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:06.494868 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:06.494874 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:06.495048 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:06.495550 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:06.495558 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:06.495564 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:06.495969 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:06.496131 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:06.496939 (+ 808us) server_negotiation.cc:299] Negotiation successful
0809 12:56:06.497214 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":300,"threads_started":1}
I20260809 12:56:06.498598 22724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:06.474498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:45659 (local address 127.21.235.65:45107)
0809 12:56:06.475284 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:06.475337 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:06.477662 (+ 2325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:06.478614 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:06.478624 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:06.479371 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:06.480239 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:06.480265 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:06.481939 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:06.481946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:06.484195 (+ 2249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:06.484210 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:06.485636 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:06.485642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:06.485889 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:06.486734 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:06.486760 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:06.489367 (+ 2607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:06.494140 (+ 4773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:06.494147 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:06.494163 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:06.494715 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:06.495171 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:06.495175 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:06.495177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:06.495401 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:06.496099 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:06.496105 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:06.496479 (+ 374us) client_negotiation.cc:769] Sending connection context
0809 12:56:06.496764 (+ 285us) client_negotiation.cc:240] Negotiation successful
0809 12:56:06.497284 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":285,"threads_started":1}
I20260809 12:56:06.500375 22721 heartbeater.cc:347] Connected to a master server at 127.21.235.126:45659
I20260809 12:56:06.500860 22721 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.21.235.66@KRBTEST.COM; defaulting to no policy
I20260809 12:56:06.501840 22721 heartbeater.cc:511] Master 127.21.235.126:45659 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.66@KRBTEST.COM" created.
I20260809 12:56:06.506129 22501 ts_manager.cc:194] Registered new tserver with Master: b2badc62acc6425aa342029ccb3f5dc3 (127.21.235.65:37665)
I20260809 12:56:06.508385 22501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:45107
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
I20260809 12:56:06.530112 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:45659
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:33403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:06.968415 22728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:06.968695 22728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:06.968792 22728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:06.976049 22728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:06.976153 22728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:06.976266 22728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:56:06.985837 22728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:45659
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:06.987112 22728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:06.988401 22728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:07.002812 22734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:07.002782 22738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:07.006781 22733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:07.008031 22728 server_base.cc:1034] running on GCE node
I20260809 12:56:07.008759 22728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:07.009877 22728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:07.011130 22728 hybrid_clock.cc:648] HybridClock initialized: now 1786280167011034 us; error 84 us; skew 500 ppm
I20260809 12:56:07.011489 22728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:07 dist-test-slave-gp6n krb5kdc[22452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280167, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:07.015331 22728 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 12:56:07.017064 22728 webserver.cc:460] Webserver started at http://127.21.235.66:46597/ using document root <none> and password file <none>
I20260809 12:56:07.017813 22728 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:07.017975 22728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:07.018307 22728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:07.021878 22728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "5a4943a9d8a643b78634f22e45d817f5"
format_stamp: "Formatted at 2026-08-09 12:56:07 on dist-test-slave-gp6n"
server_key: "3d279ad3d43af4bfb3c8e114093d04a5"
server_key_iv: "9c09b5534479fcfed5a752039cfd1b42"
server_key_version: "encryptionkey@0"
I20260809 12:56:07.022737 22728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "5a4943a9d8a643b78634f22e45d817f5"
format_stamp: "Formatted at 2026-08-09 12:56:07 on dist-test-slave-gp6n"
server_key: "3d279ad3d43af4bfb3c8e114093d04a5"
server_key_iv: "9c09b5534479fcfed5a752039cfd1b42"
server_key_version: "encryptionkey@0"
I20260809 12:56:07.028463 22728 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:56:07.032589 22745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:07.034041 22728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:56:07.034221 22728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "5a4943a9d8a643b78634f22e45d817f5"
format_stamp: "Formatted at 2026-08-09 12:56:07 on dist-test-slave-gp6n"
server_key: "3d279ad3d43af4bfb3c8e114093d04a5"
server_key_iv: "9c09b5534479fcfed5a752039cfd1b42"
server_key_version: "encryptionkey@0"
I20260809 12:56:07.034518 22728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:07.066555 22728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:07.071981 22728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:07.072324 22728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:07.073611 22728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:07.075269 22728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:07.075397 22728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:07.075577 22728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:07.075664 22728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:07.127586 22728 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:45587
I20260809 12:56:07.127704 22871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:45587 every 8 connection(s)
I20260809 12:56:07.129070 22728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:56:07.133033 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 22728
I20260809 12:56:07.133308 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:56:07.134177 22445 external_mini_cluster.cc:1428] Setting key 170db0f9fe10de9599e2cb3e23172e8f
Aug 09 12:56:07 dist-test-slave-gp6n krb5kdc[22452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280167, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:07.152823 22874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:07.132133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:52643 (local address 127.21.235.126:45659)
0809 12:56:07.132637 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:07.132642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:07.134629 (+ 1987us) server_negotiation.cc:407] Connection header received
0809 12:56:07.137453 (+ 2824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:07.137458 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:07.137544 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:07.137866 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:07.139666 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:07.140423 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:07.142045 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:07.142554 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:07.142686 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:07.146644 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:07.146649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:07.146652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:07.146684 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:07.149682 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:07.150437 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:07.150445 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:07.150449 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:07.150575 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:07.151019 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:07.151024 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:07.151027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:07.151328 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:07.151512 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:07.152233 (+ 721us) server_negotiation.cc:299] Negotiation successful
0809 12:56:07.152456 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":302,"threads_started":1}
I20260809 12:56:07.153877 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:07.133066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:45659 (local address 127.21.235.66:52643)
0809 12:56:07.134298 (+ 1232us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:07.134343 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:07.137181 (+ 2838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:07.138126 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:07.138134 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:07.138753 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:07.139484 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:07.139503 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:07.140588 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:07.140592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:07.141862 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:07.141874 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:07.142802 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:07.142810 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:07.143105 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:07.143934 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:07.143976 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:07.146331 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:07.149831 (+ 3500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:07.149841 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:07.149857 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:07.150287 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:07.150689 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:07.150694 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:07.150698 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:07.150896 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:07.151468 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:07.151479 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:07.151830 (+ 351us) client_negotiation.cc:769] Sending connection context
0809 12:56:07.152139 (+ 309us) client_negotiation.cc:240] Negotiation successful
0809 12:56:07.152691 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":998,"thread_start_us":727,"threads_started":1}
I20260809 12:56:07.155673 22872 heartbeater.cc:347] Connected to a master server at 127.21.235.126:45659
I20260809 12:56:07.155989 22872 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:07.156724 22872 heartbeater.cc:511] Master 127.21.235.126:45659 requested a full tablet report, sending...
I20260809 12:56:07.158473 22501 ts_manager.cc:194] Registered new tserver with Master: 5a4943a9d8a643b78634f22e45d817f5 (127.21.235.66:45587)
I20260809 12:56:07.160125 22501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:52643
WARNING: no policy specified for kudu/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
I20260809 12:56:07.195538 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:45659
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:33403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:56:07.512112 22721 heartbeater.cc:503] Master 127.21.235.126:45659 was elected leader, sending a full tablet report...
W20260809 12:56:07.594046 22879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:07.594380 22879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:07.594631 22879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:07.602897 22879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:07.603042 22879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:07.603236 22879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:56:07.613842 22879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:45659
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:07.615325 22879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:07.616796 22879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:07.630940 22885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:07.631615 22888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:07.631717 22886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:07.633036 22879 server_base.cc:1034] running on GCE node
I20260809 12:56:07.633689 22879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:07.634840 22879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:07.636130 22879 hybrid_clock.cc:648] HybridClock initialized: now 1786280167636038 us; error 84 us; skew 500 ppm
I20260809 12:56:07.636456 22879 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:07 dist-test-slave-gp6n krb5kdc[22452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280167, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:07.640399 22879 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 12:56:07.642117 22879 webserver.cc:460] Webserver started at http://127.21.235.67:33717/ using document root <none> and password file <none>
I20260809 12:56:07.642915 22879 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:07.643131 22879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:07.643496 22879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:07.646816 22879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "dddbbf1433d0433089f58c6ac52a5519"
format_stamp: "Formatted at 2026-08-09 12:56:07 on dist-test-slave-gp6n"
server_key: "29f15d9aaea4ec152e7b6ee8d490bde3"
server_key_iv: "e37bb024affa4160544b382f5f4ba156"
server_key_version: "encryptionkey@0"
I20260809 12:56:07.647735 22879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "dddbbf1433d0433089f58c6ac52a5519"
format_stamp: "Formatted at 2026-08-09 12:56:07 on dist-test-slave-gp6n"
server_key: "29f15d9aaea4ec152e7b6ee8d490bde3"
server_key_iv: "e37bb024affa4160544b382f5f4ba156"
server_key_version: "encryptionkey@0"
I20260809 12:56:07.653620 22879 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:56:07.657732 22896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:07.659222 22879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:56:07.659408 22879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "dddbbf1433d0433089f58c6ac52a5519"
format_stamp: "Formatted at 2026-08-09 12:56:07 on dist-test-slave-gp6n"
server_key: "29f15d9aaea4ec152e7b6ee8d490bde3"
server_key_iv: "e37bb024affa4160544b382f5f4ba156"
server_key_version: "encryptionkey@0"
I20260809 12:56:07.659660 22879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:07.687640 22879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:07.693082 22879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:07.693508 22879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:07.694885 22879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:07.696516 22879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:07.696625 22879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:07.696789 22879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:07.696909 22879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:07.752216 22879 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:46131
I20260809 12:56:07.752246 23024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:46131 every 8 connection(s)
I20260809 12:56:07.754191 22879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:56:07.757671 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 22879
I20260809 12:56:07.757893 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:56:07.758608 22445 external_mini_cluster.cc:1428] Setting key 03db77b0848ec63f045144c2feba97c9
Aug 09 12:56:07 dist-test-slave-gp6n krb5kdc[22452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280167, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:07.780097 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:07.757831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:58975 (local address 127.21.235.126:45659)
0809 12:56:07.761024 (+ 3193us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:07.761031 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:07.761052 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:56:07.763484 (+ 2432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:07.763490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:07.763632 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:07.763720 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:07.765735 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:07.766675 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:07.768258 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:07.768668 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:07.768765 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:07.773082 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:07.773087 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:07.773090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:07.773121 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:07.776518 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:07.777440 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:07.777444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:07.777447 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:07.777581 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:07.778005 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:07.778012 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:07.778019 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:07.778359 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:07.778628 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:07.779309 (+ 681us) server_negotiation.cc:299] Negotiation successful
0809 12:56:07.779544 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3066,"thread_start_us":2883,"threads_started":1}
I20260809 12:56:07.781105 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:07.758585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:45659 (local address 127.21.235.67:58975)
0809 12:56:07.759916 (+ 1331us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:07.759965 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:07.763085 (+ 3120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:07.764114 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:07.764129 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:07.764967 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:07.765561 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:07.765578 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:07.766865 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:07.766869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:07.768125 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:07.768137 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:07.768942 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:07.768948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:07.769216 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:07.770044 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:07.770071 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:07.772861 (+ 2790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:07.776653 (+ 3792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:07.776662 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:07.776680 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:07.777309 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:07.777685 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:07.777688 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:07.777691 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:07.777867 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:07.778592 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:07.778600 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:07.778939 (+ 339us) client_negotiation.cc:769] Sending connection context
0809 12:56:07.779224 (+ 285us) client_negotiation.cc:240] Negotiation successful
0809 12:56:07.779829 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1009,"thread_start_us":797,"threads_started":1}
I20260809 12:56:07.782819 23025 heartbeater.cc:347] Connected to a master server at 127.21.235.126:45659
I20260809 12:56:07.783109 23025 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:07.783823 23025 heartbeater.cc:511] Master 127.21.235.126:45659 requested a full tablet report, sending...
I20260809 12:56:07.785408 22501 ts_manager.cc:194] Registered new tserver with Master: dddbbf1433d0433089f58c6ac52a5519 (127.21.235.67:46131)
I20260809 12:56:07.786172 22501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:58975
I20260809 12:56:07.789374 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:56:07 dist-test-slave-gp6n krb5kdc[22452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280167, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 12:56:07 dist-test-slave-gp6n krb5kdc[22452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280167, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 12:56:07.854264 23034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:07.837021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58744 (local address 127.21.235.65:37665)
0809 12:56:07.838065 (+ 1044us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:07.838074 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:07.838103 (+ 29us) server_negotiation.cc:407] Connection header received
0809 12:56:07.838167 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:07.838174 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:07.838389 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:07.838661 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:07.840504 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:07.841710 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:07.843217 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:07.843895 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:07.844009 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:07.848162 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:07.848173 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:07.848188 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:07.848220 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:07.851322 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:07.852012 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:07.852021 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:07.852023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:07.852151 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:07.852555 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:07.852561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:07.852564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:07.852923 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:07.853265 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:07.853597 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 12:56:07.853878 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":886,"thread_start_us":724,"threads_started":1}
W20260809 12:56:07.855767 22624 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58744
Aug 09 12:56:07 dist-test-slave-gp6n krb5kdc[22452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 12:56:07 dist-test-slave-gp6n krb5kdc[22452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 12:56:07.933566 23034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:07.917202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58746 (local address 127.21.235.65:37665)
0809 12:56:07.917446 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:07.917453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:07.917806 (+ 353us) server_negotiation.cc:407] Connection header received
0809 12:56:07.918003 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:07.918008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:07.918083 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:07.918185 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:07.920447 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:07.921180 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:07.922547 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:07.923047 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:07.923155 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:07.927602 (+ 4447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:07.927607 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:07.927610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:07.927648 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:07.930918 (+ 3270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:07.931713 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:07.931717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:07.931720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:07.931861 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:07.932255 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:07.932258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:07.932260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:07.932523 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:07.932701 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:07.933129 (+ 428us) server_negotiation.cc:299] Negotiation successful
0809 12:56:07.933311 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260809 12:56:07.936141 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 22573
I20260809 12:56:07.951860 23034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:07.937284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58760 (local address 127.21.235.65:37665)
0809 12:56:07.937487 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:07.937491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:07.937921 (+ 430us) server_negotiation.cc:407] Connection header received
0809 12:56:07.938082 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:07.938088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:07.938161 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:07.938267 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:07.939645 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:07.940513 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:07.941868 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:07.942364 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:07.942533 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:07.945613 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:07.945619 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:07.945624 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:07.945669 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:07.948931 (+ 3262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:07.949695 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:07.949701 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:07.949705 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:07.949859 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:07.950254 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:07.950262 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:07.950268 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:07.950659 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:07.950948 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:07.951252 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 12:56:07.951491 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 12:56:07.953008 22716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:08.046083 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 22573
I20260809 12:56:08.068044 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 22728
Aug 09 12:56:08 dist-test-slave-gp6n krb5kdc[22452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.66@KRBTEST.COM
I20260809 12:56:08.085188 23043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:08.069052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33986 (local address 127.21.235.66:45587)
0809 12:56:08.069755 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:08.069766 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:08.069802 (+ 36us) server_negotiation.cc:407] Connection header received
0809 12:56:08.069891 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:08.069901 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:08.070207 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:08.070344 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:08.071647 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:08.072650 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:08.073962 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:08.074716 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:08.074815 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:08.079311 (+ 4496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:08.079322 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:08.079337 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:08.079373 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:08.082525 (+ 3152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:08.083185 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:08.083192 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:08.083194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:08.083314 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:08.083699 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:08.083702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:08.083704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:08.084053 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:08.084305 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:08.084537 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 12:56:08.084738 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":331,"threads_started":1}
I20260809 12:56:08.086838 22865 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:08.182152 22872 heartbeater.cc:503] Master 127.21.235.126:45659 was elected leader, sending a full tablet report...
I20260809 12:56:08.190663 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 22728
I20260809 12:56:08.211934 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 22879
Aug 09 12:56:08 dist-test-slave-gp6n krb5kdc[22452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.67@KRBTEST.COM
I20260809 12:56:08.235122 23045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:08.213164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59602 (local address 127.21.235.67:46131)
0809 12:56:08.214360 (+ 1196us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:08.214369 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:08.214678 (+ 309us) server_negotiation.cc:407] Connection header received
0809 12:56:08.214772 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:08.214779 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:08.215010 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:08.215225 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:08.216735 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:08.217697 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:08.219232 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:08.219795 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:08.219963 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:08.225382 (+ 5419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:08.225396 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:08.225414 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:08.225458 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:08.230498 (+ 5040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:08.231556 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:08.231565 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:08.231569 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:08.231799 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:08.232440 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:08.232445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:08.232448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:08.232919 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:08.234210 (+ 1291us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:08.234445 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 12:56:08.234715 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":347,"threads_started":1}
I20260809 12:56:08.236915 23019 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:08.326689 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 22879
I20260809 12:56:08.350661 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 22465
I20260809 12:56:08.352005 22535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:08.463728 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 22465
2026-08-09T12:56:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4703 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:56:09 dist-test-slave-gp6n krb5kdc[23052](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:56:09 dist-test-slave-gp6n krb5kdc[23052](info): set up 2 sockets
Aug 09 12:56:09 dist-test-slave-gp6n krb5kdc[23052](info): commencing operation
krb5kdc: starting...
W20260809 12:56:10.857257 22445 mini_kdc.cc:121] Time spent starting KDC: real 2.346s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:56:11 dist-test-slave-gp6n krb5kdc[23052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:56:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:56:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 12:56:11.080668 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:37011
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:37011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:11.423802 23066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:11.424100 23066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:11.424268 23066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:11.431445 23066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:56:11.431551 23066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:11.431638 23066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:11.431748 23066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:56:11.431813 23066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:56:11.441777 23066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:37011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:37011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:11.443173 23066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:11.444533 23066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:11.456887 23072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:11.456930 23073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:11.456951 23076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:11.458292 23066 server_base.cc:1034] running on GCE node
I20260809 12:56:11.459067 23066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:11.460454 23066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:11.461661 23066 hybrid_clock.cc:648] HybridClock initialized: now 1786280171461613 us; error 48 us; skew 500 ppm
I20260809 12:56:11.462018 23066 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:11 dist-test-slave-gp6n krb5kdc[23052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280171, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:11.469792 23066 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:56:11.471330 23066 webserver.cc:460] Webserver started at http://127.21.235.126:35179/ using document root <none> and password file <none>
I20260809 12:56:11.472026 23066 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:11.472178 23066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:11.472500 23066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:11.475561 23066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "c13dddaf5cb44eddb8f145f65f5c2df6"
format_stamp: "Formatted at 2026-08-09 12:56:11 on dist-test-slave-gp6n"
server_key: "e333d7a806d9296929fa08d816f9db97"
server_key_iv: "e0c77d781bcf57b596f3b3fa593fb84e"
server_key_version: "encryptionkey@0"
I20260809 12:56:11.476419 23066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "c13dddaf5cb44eddb8f145f65f5c2df6"
format_stamp: "Formatted at 2026-08-09 12:56:11 on dist-test-slave-gp6n"
server_key: "e333d7a806d9296929fa08d816f9db97"
server_key_iv: "e0c77d781bcf57b596f3b3fa593fb84e"
server_key_version: "encryptionkey@0"
I20260809 12:56:11.481833 23066 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:56:11.485620 23084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:11.487044 23066 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:56:11.487246 23066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "c13dddaf5cb44eddb8f145f65f5c2df6"
format_stamp: "Formatted at 2026-08-09 12:56:11 on dist-test-slave-gp6n"
server_key: "e333d7a806d9296929fa08d816f9db97"
server_key_iv: "e0c77d781bcf57b596f3b3fa593fb84e"
server_key_version: "encryptionkey@0"
I20260809 12:56:11.487471 23066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:11.501642 23066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:11.506119 23066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:11.506464 23066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:11.525851 23066 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:37011
I20260809 12:56:11.525888 23141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:37011 every 8 connection(s)
I20260809 12:56:11.527248 23066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:56:11.529408 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 23066
I20260809 12:56:11.529683 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:56:11.530617 22445 external_mini_cluster.cc:1428] Setting key c919fd822cf3034303d022f23cd3f1bd
I20260809 12:56:11.532397 23142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 12:56:11 dist-test-slave-gp6n krb5kdc[23052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:11.545635 23142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6: Bootstrap starting.
I20260809 12:56:11.549288 23142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:11.550773 23142 log.cc:824] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:11.553876 23145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:11.532830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60794 (local address 127.21.235.126:37011)
0809 12:56:11.533609 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:11.533622 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:11.533663 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:56:11.535324 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:11.535346 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:11.535745 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:11.536074 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:11.538144 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:11.539556 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:11.540849 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:11.541532 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:11.541723 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:11.545729 (+ 4006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:11.545737 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:11.545751 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:11.545821 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:11.548874 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:11.549584 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:11.549593 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:11.549601 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:11.549861 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:11.550246 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:11.550252 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:11.550256 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:11.551048 (+ 792us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:11.551447 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:11.551869 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 12:56:11.552443 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":361,"threads_started":1}
I20260809 12:56:11.554663 23142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6: No bootstrap required, opened a new log
I20260809 12:56:11.563094 23142 raft_consensus.cc:348] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13dddaf5cb44eddb8f145f65f5c2df6" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 37011 } }
I20260809 12:56:11.563484 23142 raft_consensus.cc:374] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:11.563563 23142 raft_consensus.cc:729] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c13dddaf5cb44eddb8f145f65f5c2df6, State: Initialized, Role: FOLLOWER
I20260809 12:56:11.564155 23142 consensus_queue.cc:260] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13dddaf5cb44eddb8f145f65f5c2df6" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 37011 } }
I20260809 12:56:11.564376 23142 raft_consensus.cc:388] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:56:11.564487 23142 raft_consensus.cc:482] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:56:11.564625 23142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:11.567070 23142 raft_consensus.cc:504] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13dddaf5cb44eddb8f145f65f5c2df6" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 37011 } }
I20260809 12:56:11.567623 23142 leader_election.cc:302] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c13dddaf5cb44eddb8f145f65f5c2df6; no voters:
I20260809 12:56:11.568248 23142 leader_election.cc:288] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:56:11.568408 23147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:11.569180 23147 raft_consensus.cc:686] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6 [term 1 LEADER]: Becoming Leader. State: Replica: c13dddaf5cb44eddb8f145f65f5c2df6, State: Running, Role: LEADER
I20260809 12:56:11.569808 23147 consensus_queue.cc:237] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13dddaf5cb44eddb8f145f65f5c2df6" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 37011 } }
I20260809 12:56:11.570554 23142 sys_catalog.cc:564] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:56:11.572722 23148 sys_catalog.cc:455] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c13dddaf5cb44eddb8f145f65f5c2df6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13dddaf5cb44eddb8f145f65f5c2df6" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 37011 } } }
I20260809 12:56:11.573086 23148 sys_catalog.cc:458] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6 [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:11.573662 23149 sys_catalog.cc:455] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c13dddaf5cb44eddb8f145f65f5c2df6. Latest consensus state: current_term: 1 leader_uuid: "c13dddaf5cb44eddb8f145f65f5c2df6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13dddaf5cb44eddb8f145f65f5c2df6" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 37011 } } }
I20260809 12:56:11.574054 23149 sys_catalog.cc:458] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6 [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:11.576095 23154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:56:11.581346 23154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:56:11.590672 23154 catalog_manager.cc:1347] Generated new cluster ID: 016afaec88a14049933141324093dd13
I20260809 12:56:11.590791 23154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:56:11.611608 23154 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:56:11.612965 23154 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:56:11.621250 23154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c13dddaf5cb44eddb8f145f65f5c2df6: Generated new TSK 0
I20260809 12:56:11.622135 23154 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
I20260809 12:56:11.682767 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:37011
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:12.027935 23168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:12.028216 23168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:12.028312 23168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:12.035502 23168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:12.035614 23168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:12.035732 23168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:56:12.045558 23168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:37011
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:12.046844 23168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:12.048144 23168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:12.062386 23174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:12.062341 23175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:12.062620 23177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:12.063140 23168 server_base.cc:1034] running on GCE node
I20260809 12:56:12.063781 23168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:12.064770 23168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:12.066000 23168 hybrid_clock.cc:648] HybridClock initialized: now 1786280172065925 us; error 90 us; skew 500 ppm
I20260809 12:56:12.066330 23168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:12 dist-test-slave-gp6n krb5kdc[23052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280172, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:12.069692 23168 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 12:56:12.071416 23168 webserver.cc:460] Webserver started at http://127.21.235.65:42547/ using document root <none> and password file <none>
I20260809 12:56:12.072117 23168 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:12.072250 23168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:12.072604 23168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:12.075647 23168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "7b6e1b11f106448f8bdf0be637969f5a"
format_stamp: "Formatted at 2026-08-09 12:56:12 on dist-test-slave-gp6n"
server_key: "ac3bd70e399d4a69974051922e15025f"
server_key_iv: "66675cf422bad965a48afa4de662e059"
server_key_version: "encryptionkey@0"
I20260809 12:56:12.076524 23168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "7b6e1b11f106448f8bdf0be637969f5a"
format_stamp: "Formatted at 2026-08-09 12:56:12 on dist-test-slave-gp6n"
server_key: "ac3bd70e399d4a69974051922e15025f"
server_key_iv: "66675cf422bad965a48afa4de662e059"
server_key_version: "encryptionkey@0"
I20260809 12:56:12.081723 23168 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:56:12.085340 23185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:12.086846 23168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:56:12.087136 23168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "7b6e1b11f106448f8bdf0be637969f5a"
format_stamp: "Formatted at 2026-08-09 12:56:12 on dist-test-slave-gp6n"
server_key: "ac3bd70e399d4a69974051922e15025f"
server_key_iv: "66675cf422bad965a48afa4de662e059"
server_key_version: "encryptionkey@0"
I20260809 12:56:12.087393 23168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:12.109851 23168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:12.114517 23168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:12.114821 23168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:12.115813 23168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:12.117146 23168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:12.117249 23168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:12.117460 23168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:12.117542 23168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:12.155712 23168 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:38271
I20260809 12:56:12.155761 23316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:38271 every 8 connection(s)
I20260809 12:56:12.157164 23168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:56:12.163275 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 23168
I20260809 12:56:12.163523 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:56:12.164201 22445 external_mini_cluster.cc:1428] Setting key 8611fd2413b76043bd6a7bb8043f2875
Aug 09 12:56:12 dist-test-slave-gp6n krb5kdc[23052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280172, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:12.178184 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:12.159906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:50923 (local address 127.21.235.126:37011)
0809 12:56:12.160346 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:12.160351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:12.161394 (+ 1043us) server_negotiation.cc:407] Connection header received
0809 12:56:12.163301 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:12.163308 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:12.163398 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:12.163563 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:12.165121 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:12.165873 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:12.167506 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:12.167904 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:12.168040 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:12.171743 (+ 3703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:12.171748 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:12.171751 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:12.171782 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:12.174639 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:12.175621 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:12.175633 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:12.175635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:12.175780 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:12.176310 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:12.176314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:12.176316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:12.176677 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:12.176856 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:12.177718 (+ 862us) server_negotiation.cc:299] Negotiation successful
0809 12:56:12.177902 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":216,"threads_started":1}
I20260809 12:56:12.179831 23321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:12.160518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:37011 (local address 127.21.235.65:50923)
0809 12:56:12.161222 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:12.161259 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:12.163022 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:12.163792 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:12.163800 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:12.164364 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:12.164938 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:12.164965 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:12.166055 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:12.166060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:12.167355 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:12.167367 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:12.168131 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:12.168141 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:12.168368 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:12.169103 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:12.169124 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:12.171482 (+ 2358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:12.174777 (+ 3295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:12.174789 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:12.174808 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:12.175482 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:12.175892 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:12.175900 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:12.175906 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:12.176179 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:12.176827 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:12.176838 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:12.177289 (+ 451us) client_negotiation.cc:769] Sending connection context
0809 12:56:12.177591 (+ 302us) client_negotiation.cc:240] Negotiation successful
0809 12:56:12.178232 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":274,"threads_started":1}
I20260809 12:56:12.181581 23317 heartbeater.cc:347] Connected to a master server at 127.21.235.126:37011
I20260809 12:56:12.181931 23317 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:12.182770 23317 heartbeater.cc:511] Master 127.21.235.126:37011 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.235.66@KRBTEST.COM; defaulting to no policy
I20260809 12:56:12.185098 23102 ts_manager.cc:194] Registered new tserver with Master: 7b6e1b11f106448f8bdf0be637969f5a (127.21.235.65:38271)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.66@KRBTEST.COM" created.
I20260809 12:56:12.187058 23102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:50923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
I20260809 12:56:12.211735 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:37011
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:12.560758 23326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:12.561012 23326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:12.561169 23326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:12.568136 23326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:12.568243 23326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:12.568472 23326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:56:12.578485 23326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:37011
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:12.579769 23326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:12.581097 23326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:12.594369 23332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:12.594475 23333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:12.594455 23335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:12.596024 23326 server_base.cc:1034] running on GCE node
I20260809 12:56:12.596602 23326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:12.597501 23326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:12.598701 23326 hybrid_clock.cc:648] HybridClock initialized: now 1786280172598659 us; error 30 us; skew 500 ppm
I20260809 12:56:12.599020 23326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:12 dist-test-slave-gp6n krb5kdc[23052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280172, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:12.602241 23326 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 12:56:12.603758 23326 webserver.cc:460] Webserver started at http://127.21.235.66:35481/ using document root <none> and password file <none>
I20260809 12:56:12.604458 23326 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:12.604589 23326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:12.604918 23326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:12.608006 23326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "4263e9adaebd4eaab9f38e2f0aed48f0"
format_stamp: "Formatted at 2026-08-09 12:56:12 on dist-test-slave-gp6n"
server_key: "65a48478b9ae6f9703e654dbfc1835ee"
server_key_iv: "efcd009220515bd8284d30390c4ed452"
server_key_version: "encryptionkey@0"
I20260809 12:56:12.608811 23326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "4263e9adaebd4eaab9f38e2f0aed48f0"
format_stamp: "Formatted at 2026-08-09 12:56:12 on dist-test-slave-gp6n"
server_key: "65a48478b9ae6f9703e654dbfc1835ee"
server_key_iv: "efcd009220515bd8284d30390c4ed452"
server_key_version: "encryptionkey@0"
I20260809 12:56:12.613845 23326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:56:12.617370 23344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:12.618813 23326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:56:12.618993 23326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "4263e9adaebd4eaab9f38e2f0aed48f0"
format_stamp: "Formatted at 2026-08-09 12:56:12 on dist-test-slave-gp6n"
server_key: "65a48478b9ae6f9703e654dbfc1835ee"
server_key_iv: "efcd009220515bd8284d30390c4ed452"
server_key_version: "encryptionkey@0"
I20260809 12:56:12.619189 23326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:12.638985 23326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:12.643370 23326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:12.643664 23326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:12.644649 23326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:12.646003 23326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:12.646108 23326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:12.646268 23326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:12.646343 23326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:12.680853 23326 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:37761
I20260809 12:56:12.680907 23477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:37761 every 8 connection(s)
I20260809 12:56:12.682166 23326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:56:12.690873 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 23326
I20260809 12:56:12.691108 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:56:12.691659 22445 external_mini_cluster.cc:1428] Setting key 4f8eae52938445bd29cc7ef1d6321fc4
Aug 09 12:56:12 dist-test-slave-gp6n krb5kdc[23052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280172, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:12.703049 23480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:12.684839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:46197 (local address 127.21.235.126:37011)
0809 12:56:12.685329 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:12.685337 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:12.686359 (+ 1022us) server_negotiation.cc:407] Connection header received
0809 12:56:12.688269 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:12.688274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:12.688343 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:12.688442 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:12.689894 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:12.690735 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:12.693088 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:12.693592 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:12.693729 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:12.697429 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:12.697434 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:12.697438 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:12.697468 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:12.700015 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:12.700744 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:12.700752 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:12.700757 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:12.700891 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:12.701321 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:12.701329 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:12.701333 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:12.701593 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:12.701750 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:12.702394 (+ 644us) server_negotiation.cc:299] Negotiation successful
0809 12:56:12.702645 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":229,"threads_started":1}
I20260809 12:56:12.703940 23482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:12.685397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:37011 (local address 127.21.235.66:46197)
0809 12:56:12.686177 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:12.686223 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:12.688020 (+ 1797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:12.688649 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:12.688657 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:12.689168 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:12.689737 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:12.689754 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:12.690920 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:12.690926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:12.692925 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:12.692938 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:12.693852 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:12.693864 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:12.694084 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:12.694787 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:12.694828 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:12.697198 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:12.700157 (+ 2959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:12.700165 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:12.700181 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:12.700618 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:12.701007 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:12.701011 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:12.701015 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:12.701205 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:12.701723 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:12.701731 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:12.702054 (+ 323us) client_negotiation.cc:769] Sending connection context
0809 12:56:12.702329 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 12:56:12.702856 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":342,"threads_started":1}
I20260809 12:56:12.705597 23478 heartbeater.cc:347] Connected to a master server at 127.21.235.126:37011
I20260809 12:56:12.705904 23478 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:12.706672 23478 heartbeater.cc:511] Master 127.21.235.126:37011 requested a full tablet report, sending...
I20260809 12:56:12.708163 23102 ts_manager.cc:194] Registered new tserver with Master: 4263e9adaebd4eaab9f38e2f0aed48f0 (127.21.235.66:37761)
I20260809 12:56:12.709069 23102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:46197
WARNING: no policy specified for kudu/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
I20260809 12:56:12.737327 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:37011
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:13.067204 23485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:13.067487 23485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:13.067585 23485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:13.074962 23485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:13.075070 23485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:13.075191 23485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:56:13.085569 23485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:37011
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:13.086855 23485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:13.088220 23485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:13.102963 23492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:13.102882 23493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:13.103093 23495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:13.104300 23485 server_base.cc:1034] running on GCE node
I20260809 12:56:13.105104 23485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:13.106176 23485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:13.107460 23485 hybrid_clock.cc:648] HybridClock initialized: now 1786280173107384 us; error 82 us; skew 500 ppm
I20260809 12:56:13.107824 23485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:13 dist-test-slave-gp6n krb5kdc[23052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280173, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:13.111233 23485 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 12:56:13.112660 23485 webserver.cc:460] Webserver started at http://127.21.235.67:43413/ using document root <none> and password file <none>
I20260809 12:56:13.113284 23485 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:13.113386 23485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:13.113641 23485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:13.116562 23485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "656d873850e54e4fbe5441651a43e253"
format_stamp: "Formatted at 2026-08-09 12:56:13 on dist-test-slave-gp6n"
server_key: "48ae24f204b392e3cee6aa637fec0305"
server_key_iv: "0bb20e8b012f9e08d5af9158a3ad011d"
server_key_version: "encryptionkey@0"
I20260809 12:56:13.117311 23485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "656d873850e54e4fbe5441651a43e253"
format_stamp: "Formatted at 2026-08-09 12:56:13 on dist-test-slave-gp6n"
server_key: "48ae24f204b392e3cee6aa637fec0305"
server_key_iv: "0bb20e8b012f9e08d5af9158a3ad011d"
server_key_version: "encryptionkey@0"
I20260809 12:56:13.122575 23485 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 12:56:13.126655 23502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:13.128106 23485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:56:13.128387 23485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "656d873850e54e4fbe5441651a43e253"
format_stamp: "Formatted at 2026-08-09 12:56:13 on dist-test-slave-gp6n"
server_key: "48ae24f204b392e3cee6aa637fec0305"
server_key_iv: "0bb20e8b012f9e08d5af9158a3ad011d"
server_key_version: "encryptionkey@0"
I20260809 12:56:13.128605 23485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:13.149441 23485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:13.153911 23485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:13.154225 23485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:13.155423 23485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:13.156915 23485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:13.156991 23485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:13.157121 23485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:13.157169 23485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:13.190079 23317 heartbeater.cc:503] Master 127.21.235.126:37011 was elected leader, sending a full tablet report...
I20260809 12:56:13.192165 23485 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:45427
I20260809 12:56:13.192225 23636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:45427 every 8 connection(s)
I20260809 12:56:13.193583 23485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:56:13.196563 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 23485
I20260809 12:56:13.196787 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:56:13.197362 22445 external_mini_cluster.cc:1428] Setting key 62840ed82e99b8c9e4cc804955c6292f
Aug 09 12:56:13 dist-test-slave-gp6n krb5kdc[23052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280173, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:13.215943 23480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.196242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:58111 (local address 127.21.235.126:37011)
0809 12:56:13.196424 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:13.196428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:13.198034 (+ 1606us) server_negotiation.cc:407] Connection header received
0809 12:56:13.200873 (+ 2839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:13.200880 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:13.200973 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:13.201064 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:13.202607 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.203507 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.205283 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.205739 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.205867 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.209404 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:13.209409 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:13.209413 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:13.209446 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:13.213096 (+ 3650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:13.213807 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:13.213811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:13.213813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:13.213933 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:13.214371 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:13.214374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:13.214376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:13.214697 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:13.214830 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:13.215523 (+ 693us) server_negotiation.cc:299] Negotiation successful
0809 12:56:13.215691 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 12:56:13.216936 23639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.196989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:37011 (local address 127.21.235.67:58111)
0809 12:56:13.197841 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:13.197890 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:13.200531 (+ 2641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:13.201274 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:13.201282 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:13.201798 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:13.202383 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:13.202450 (+ 67us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.203654 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.203658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:13.205135 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:13.205147 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.205979 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.205984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:13.206171 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.206877 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:13.206899 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:13.209183 (+ 2284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:13.213242 (+ 4059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:13.213249 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:13.213262 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:13.213690 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:13.214025 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:13.214028 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:13.214030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:13.214267 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:13.214836 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:13.214842 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:13.215182 (+ 340us) client_negotiation.cc:769] Sending connection context
0809 12:56:13.215414 (+ 232us) client_negotiation.cc:240] Negotiation successful
0809 12:56:13.215807 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":379,"threads_started":1}
I20260809 12:56:13.218657 23637 heartbeater.cc:347] Connected to a master server at 127.21.235.126:37011
I20260809 12:56:13.219002 23637 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:13.219766 23637 heartbeater.cc:511] Master 127.21.235.126:37011 requested a full tablet report, sending...
I20260809 12:56:13.221323 23102 ts_manager.cc:194] Registered new tserver with Master: 656d873850e54e4fbe5441651a43e253 (127.21.235.67:45427)
I20260809 12:56:13.222132 23102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:58111
I20260809 12:56:13.225672 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:56:13.247331 23480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.233750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60808 (local address 127.21.235.126:37011)
0809 12:56:13.234035 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:13.234039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:13.234615 (+ 576us) server_negotiation.cc:407] Connection header received
0809 12:56:13.234764 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:13.234768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:13.234840 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:13.234928 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:13.236628 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.237425 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.238979 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.239370 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.239466 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.241917 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:13.241922 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:13.241925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:13.241961 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:13.244596 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:13.245324 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:13.245328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:13.245330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:13.245455 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:13.245898 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:13.245901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:13.245903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:13.246276 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:13.246536 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:13.246892 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 12:56:13.247064 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260809 12:56:13.257004 23102 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:56:13.265805 23102 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:56:13.305609 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.283032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:37761 (local address 127.0.0.1:42476)
0809 12:56:13.283975 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:13.284024 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:13.284293 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:13.288556 (+ 4263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:13.288568 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:13.288612 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:13.290577 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:13.290599 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.295066 (+ 4467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.295075 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:13.300831 (+ 5756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:13.300858 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.304484 (+ 3626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.304493 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:13.304734 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.304798 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 12:56:13.304929 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 12:56:13.305174 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":20766,"threads_started":1}
I20260809 12:56:13.307438 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.283592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42476 (local address 127.21.235.66:37761)
0809 12:56:13.287725 (+ 4133us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:13.287739 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:13.287778 (+ 39us) server_negotiation.cc:407] Connection header received
0809 12:56:13.287908 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:13.287919 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:13.288242 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:13.290919 (+ 2677us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:13.291218 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.294863 (+ 3645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.301114 (+ 6251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.303437 (+ 2323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.306545 (+ 3108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.306825 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:13.306934 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 12:56:13.307121 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3844,"thread_start_us":7237,"threads_started":1}
I20260809 12:56:13.312927 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.283088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:38271 (local address 127.0.0.1:50934)
0809 12:56:13.304136 (+ 21048us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:13.304160 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:13.304381 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:13.305888 (+ 1507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:13.305892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:13.305916 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:13.306390 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:13.306470 (+ 80us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.308835 (+ 2365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.308851 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:13.310596 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:13.310611 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.311980 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.311986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:13.312209 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.312269 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 12:56:13.312357 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 12:56:13.312485 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20880,"mutex_wait_us":26,"thread_start_us":679,"threads_started":1}
I20260809 12:56:13.313207 23652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.283603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50934 (local address 127.21.235.65:38271)
0809 12:56:13.304989 (+ 21386us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:13.305001 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:13.305040 (+ 39us) server_negotiation.cc:407] Connection header received
0809 12:56:13.305202 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:13.305209 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:13.305485 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:13.306673 (+ 1188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:13.307084 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.308658 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.310801 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.311736 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.311886 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.312144 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:13.312478 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 12:56:13.312668 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21187,"thread_start_us":20736,"threads_started":1}
I20260809 12:56:13.312431 23398 tablet_service.cc:1467] Processing CreateTablet for tablet d1f4d0ee3f4d4378b9424a6fa4fcb31e (DEFAULT_TABLE table=test-table [id=2fbe588b24024a36add6d0bfe83c9771]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:13.314929 23398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1f4d0ee3f4d4378b9424a6fa4fcb31e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:13.315574 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.288293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49346 (local address 127.21.235.67:45427)
0809 12:56:13.304988 (+ 16695us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:13.304997 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:13.306170 (+ 1173us) server_negotiation.cc:407] Connection header received
0809 12:56:13.306327 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:13.306337 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:13.306667 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:13.306986 (+ 319us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:13.307798 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.309420 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.312479 (+ 3059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.314130 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.314452 (+ 322us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.314693 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:13.314930 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 12:56:13.315116 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16510,"thread_start_us":16355,"threads_started":1}
I20260809 12:56:13.315591 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.287475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:45427 (local address 127.0.0.1:49346)
0809 12:56:13.305696 (+ 18221us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:13.305718 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:13.306032 (+ 314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:13.307023 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:13.307030 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:13.307061 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:13.307561 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:13.307575 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.309626 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.309635 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:13.312269 (+ 2634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:13.312286 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.314428 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.314436 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:13.314623 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.314669 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 12:56:13.314886 (+ 217us) client_negotiation.cc:240] Negotiation successful
0809 12:56:13.315033 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18066,"mutex_wait_us":6,"thread_start_us":17696,"threads_started":1}
I20260809 12:56:13.317382 23242 tablet_service.cc:1467] Processing CreateTablet for tablet d1f4d0ee3f4d4378b9424a6fa4fcb31e (DEFAULT_TABLE table=test-table [id=2fbe588b24024a36add6d0bfe83c9771]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:13.319100 23242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1f4d0ee3f4d4378b9424a6fa4fcb31e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:13.318930 23559 tablet_service.cc:1467] Processing CreateTablet for tablet d1f4d0ee3f4d4378b9424a6fa4fcb31e (DEFAULT_TABLE table=test-table [id=2fbe588b24024a36add6d0bfe83c9771]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:13.320210 23559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1f4d0ee3f4d4378b9424a6fa4fcb31e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:13.334575 23655 tablet_bootstrap.cc:492] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 656d873850e54e4fbe5441651a43e253: Bootstrap starting.
I20260809 12:56:13.335572 23656 tablet_bootstrap.cc:492] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 4263e9adaebd4eaab9f38e2f0aed48f0: Bootstrap starting.
I20260809 12:56:13.337427 23657 tablet_bootstrap.cc:492] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a: Bootstrap starting.
I20260809 12:56:13.339841 23656 tablet_bootstrap.cc:654] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 4263e9adaebd4eaab9f38e2f0aed48f0: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:13.339883 23655 tablet_bootstrap.cc:654] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 656d873850e54e4fbe5441651a43e253: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:13.340935 23657 tablet_bootstrap.cc:654] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:13.341590 23656 log.cc:824] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 4263e9adaebd4eaab9f38e2f0aed48f0: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:13.341568 23655 log.cc:824] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 656d873850e54e4fbe5441651a43e253: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:13.342151 23657 log.cc:824] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:13.345208 23656 tablet_bootstrap.cc:492] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 4263e9adaebd4eaab9f38e2f0aed48f0: No bootstrap required, opened a new log
I20260809 12:56:13.345543 23656 ts_tablet_manager.cc:1397] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 4263e9adaebd4eaab9f38e2f0aed48f0: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260809 12:56:13.345781 23657 tablet_bootstrap.cc:492] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a: No bootstrap required, opened a new log
I20260809 12:56:13.346060 23657 ts_tablet_manager.cc:1397] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 12:56:13.346223 23655 tablet_bootstrap.cc:492] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 656d873850e54e4fbe5441651a43e253: No bootstrap required, opened a new log
I20260809 12:56:13.346545 23655 ts_tablet_manager.cc:1397] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 656d873850e54e4fbe5441651a43e253: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260809 12:56:13.353941 23656 raft_consensus.cc:348] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 4263e9adaebd4eaab9f38e2f0aed48f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4263e9adaebd4eaab9f38e2f0aed48f0" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37761 } } peers { permanent_uuid: "7b6e1b11f106448f8bdf0be637969f5a" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38271 } } peers { permanent_uuid: "656d873850e54e4fbe5441651a43e253" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 45427 } }
I20260809 12:56:13.354380 23656 raft_consensus.cc:374] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 4263e9adaebd4eaab9f38e2f0aed48f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:13.354558 23656 raft_consensus.cc:729] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 4263e9adaebd4eaab9f38e2f0aed48f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4263e9adaebd4eaab9f38e2f0aed48f0, State: Initialized, Role: FOLLOWER
I20260809 12:56:13.355255 23656 consensus_queue.cc:260] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 4263e9adaebd4eaab9f38e2f0aed48f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4263e9adaebd4eaab9f38e2f0aed48f0" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37761 } } peers { permanent_uuid: "7b6e1b11f106448f8bdf0be637969f5a" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38271 } } peers { permanent_uuid: "656d873850e54e4fbe5441651a43e253" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 45427 } }
I20260809 12:56:13.356978 23478 heartbeater.cc:503] Master 127.21.235.126:37011 was elected leader, sending a full tablet report...
I20260809 12:56:13.357836 23656 ts_tablet_manager.cc:1428] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 4263e9adaebd4eaab9f38e2f0aed48f0: Time spent starting tablet: real 0.012s user 0.007s sys 0.003s
I20260809 12:56:13.358906 23657 raft_consensus.cc:348] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4263e9adaebd4eaab9f38e2f0aed48f0" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37761 } } peers { permanent_uuid: "7b6e1b11f106448f8bdf0be637969f5a" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38271 } } peers { permanent_uuid: "656d873850e54e4fbe5441651a43e253" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 45427 } }
I20260809 12:56:13.359416 23657 raft_consensus.cc:374] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:13.359160 23655 raft_consensus.cc:348] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 656d873850e54e4fbe5441651a43e253 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4263e9adaebd4eaab9f38e2f0aed48f0" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37761 } } peers { permanent_uuid: "7b6e1b11f106448f8bdf0be637969f5a" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38271 } } peers { permanent_uuid: "656d873850e54e4fbe5441651a43e253" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 45427 } }
I20260809 12:56:13.359593 23657 raft_consensus.cc:729] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b6e1b11f106448f8bdf0be637969f5a, State: Initialized, Role: FOLLOWER
I20260809 12:56:13.359658 23655 raft_consensus.cc:374] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 656d873850e54e4fbe5441651a43e253 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:13.359760 23655 raft_consensus.cc:729] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 656d873850e54e4fbe5441651a43e253 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 656d873850e54e4fbe5441651a43e253, State: Initialized, Role: FOLLOWER
I20260809 12:56:13.360332 23657 consensus_queue.cc:260] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4263e9adaebd4eaab9f38e2f0aed48f0" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37761 } } peers { permanent_uuid: "7b6e1b11f106448f8bdf0be637969f5a" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38271 } } peers { permanent_uuid: "656d873850e54e4fbe5441651a43e253" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 45427 } }
I20260809 12:56:13.360502 23655 consensus_queue.cc:260] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 656d873850e54e4fbe5441651a43e253 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4263e9adaebd4eaab9f38e2f0aed48f0" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37761 } } peers { permanent_uuid: "7b6e1b11f106448f8bdf0be637969f5a" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38271 } } peers { permanent_uuid: "656d873850e54e4fbe5441651a43e253" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 45427 } }
I20260809 12:56:13.362708 23657 ts_tablet_manager.cc:1428] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a: Time spent starting tablet: real 0.017s user 0.013s sys 0.005s
I20260809 12:56:13.363934 23637 heartbeater.cc:503] Master 127.21.235.126:37011 was elected leader, sending a full tablet report...
I20260809 12:56:13.364925 23655 ts_tablet_manager.cc:1428] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 656d873850e54e4fbe5441651a43e253: Time spent starting tablet: real 0.018s user 0.013s sys 0.003s
I20260809 12:56:13.365795 23664 raft_consensus.cc:482] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:56:13.366271 23664 raft_consensus.cc:504] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4263e9adaebd4eaab9f38e2f0aed48f0" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37761 } } peers { permanent_uuid: "7b6e1b11f106448f8bdf0be637969f5a" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38271 } } peers { permanent_uuid: "656d873850e54e4fbe5441651a43e253" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 45427 } }
I20260809 12:56:13.369052 23664 leader_election.cc:288] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4263e9adaebd4eaab9f38e2f0aed48f0 (127.21.235.66:37761), 656d873850e54e4fbe5441651a43e253 (127.21.235.67:45427)
I20260809 12:56:13.377961 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.369588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:38355 (local address 127.21.235.66:37761)
0809 12:56:13.369831 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:13.369841 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:13.370167 (+ 326us) server_negotiation.cc:407] Connection header received
0809 12:56:13.370531 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:13.370538 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:13.370617 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:13.370691 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:13.371751 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.373046 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.375667 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.376964 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.377129 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.377182 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:13.377683 (+ 501us) server_negotiation.cc:299] Negotiation successful
0809 12:56:13.377774 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 12:56:13.378366 23665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.369210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:37761 (local address 127.21.235.65:38355)
0809 12:56:13.369923 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:13.369950 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:13.370308 (+ 358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:13.370906 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:13.370912 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:13.370935 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:13.371513 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:13.371531 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.373201 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.373205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:13.375504 (+ 2299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:13.375518 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.377206 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.377214 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:13.377484 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.377528 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 12:56:13.377668 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 12:56:13.377807 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":367,"threads_started":1}
I20260809 12:56:13.379562 23666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.370083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:45427 (local address 127.21.235.65:50699)
0809 12:56:13.371983 (+ 1900us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:13.372003 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:13.372189 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:13.372547 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:13.372553 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:13.372580 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:13.373154 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:13.373168 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.374815 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.374819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:13.376925 (+ 2106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:13.376943 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.379007 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.379015 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:13.379159 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.379183 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 12:56:13.379262 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 12:56:13.379344 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1727,"thread_start_us":1560,"threads_started":1}
I20260809 12:56:13.379910 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.370166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:50699 (local address 127.21.235.67:45427)
0809 12:56:13.370461 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:13.370470 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:13.372101 (+ 1631us) server_negotiation.cc:407] Connection header received
0809 12:56:13.372343 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:13.372349 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:13.372428 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:13.372527 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:13.373359 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.374669 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.377136 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.378790 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.378923 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.378973 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:13.379355 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 12:56:13.379535 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:56:13.379908 23422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1f4d0ee3f4d4378b9424a6fa4fcb31e" candidate_uuid: "7b6e1b11f106448f8bdf0be637969f5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4263e9adaebd4eaab9f38e2f0aed48f0" is_pre_election: true
I20260809 12:56:13.380759 23422 raft_consensus.cc:2454] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 4263e9adaebd4eaab9f38e2f0aed48f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b6e1b11f106448f8bdf0be637969f5a in term 0.
I20260809 12:56:13.380810 23582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1f4d0ee3f4d4378b9424a6fa4fcb31e" candidate_uuid: "7b6e1b11f106448f8bdf0be637969f5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "656d873850e54e4fbe5441651a43e253" is_pre_election: true
I20260809 12:56:13.381409 23582 raft_consensus.cc:2454] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 656d873850e54e4fbe5441651a43e253 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b6e1b11f106448f8bdf0be637969f5a in term 0.
I20260809 12:56:13.381736 23189 leader_election.cc:302] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4263e9adaebd4eaab9f38e2f0aed48f0, 7b6e1b11f106448f8bdf0be637969f5a; no voters:
I20260809 12:56:13.382287 23664 raft_consensus.cc:2781] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:56:13.382477 23664 raft_consensus.cc:482] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:56:13.382603 23664 raft_consensus.cc:3037] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:13.385102 23664 raft_consensus.cc:504] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4263e9adaebd4eaab9f38e2f0aed48f0" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37761 } } peers { permanent_uuid: "7b6e1b11f106448f8bdf0be637969f5a" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38271 } } peers { permanent_uuid: "656d873850e54e4fbe5441651a43e253" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 45427 } }
I20260809 12:56:13.385696 23664 leader_election.cc:288] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a [CANDIDATE]: Term 1 election: Requested vote from peers 4263e9adaebd4eaab9f38e2f0aed48f0 (127.21.235.66:37761), 656d873850e54e4fbe5441651a43e253 (127.21.235.67:45427)
I20260809 12:56:13.386264 23582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1f4d0ee3f4d4378b9424a6fa4fcb31e" candidate_uuid: "7b6e1b11f106448f8bdf0be637969f5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "656d873850e54e4fbe5441651a43e253"
I20260809 12:56:13.386265 23422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1f4d0ee3f4d4378b9424a6fa4fcb31e" candidate_uuid: "7b6e1b11f106448f8bdf0be637969f5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4263e9adaebd4eaab9f38e2f0aed48f0"
I20260809 12:56:13.386541 23422 raft_consensus.cc:3037] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 4263e9adaebd4eaab9f38e2f0aed48f0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:13.386541 23582 raft_consensus.cc:3037] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 656d873850e54e4fbe5441651a43e253 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:13.390540 23422 raft_consensus.cc:2454] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 4263e9adaebd4eaab9f38e2f0aed48f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b6e1b11f106448f8bdf0be637969f5a in term 1.
I20260809 12:56:13.390775 23582 raft_consensus.cc:2454] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 656d873850e54e4fbe5441651a43e253 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b6e1b11f106448f8bdf0be637969f5a in term 1.
I20260809 12:56:13.391140 23189 leader_election.cc:302] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4263e9adaebd4eaab9f38e2f0aed48f0, 7b6e1b11f106448f8bdf0be637969f5a; no voters:
I20260809 12:56:13.391476 23664 raft_consensus.cc:2781] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:13.391733 23664 raft_consensus.cc:686] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a [term 1 LEADER]: Becoming Leader. State: Replica: 7b6e1b11f106448f8bdf0be637969f5a, State: Running, Role: LEADER
I20260809 12:56:13.392222 23664 consensus_queue.cc:237] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4263e9adaebd4eaab9f38e2f0aed48f0" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37761 } } peers { permanent_uuid: "7b6e1b11f106448f8bdf0be637969f5a" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38271 } } peers { permanent_uuid: "656d873850e54e4fbe5441651a43e253" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 45427 } }
I20260809 12:56:13.397599 23102 catalog_manager.cc:5495] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b6e1b11f106448f8bdf0be637969f5a (127.21.235.65). New cstate: current_term: 1 leader_uuid: "7b6e1b11f106448f8bdf0be637969f5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4263e9adaebd4eaab9f38e2f0aed48f0" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b6e1b11f106448f8bdf0be637969f5a" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "656d873850e54e4fbe5441651a43e253" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 45427 } health_report { overall_health: UNKNOWN } } }
W20260809 12:56:13.409727 23318 tablet.cc:2369] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 7b6e1b11f106448f8bdf0be637969f5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 12:56:13 dist-test-slave-gp6n krb5kdc[23052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 12:56:13.426906 23652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.409846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50936 (local address 127.21.235.65:38271)
0809 12:56:13.410106 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:13.410111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:13.410904 (+ 793us) server_negotiation.cc:407] Connection header received
0809 12:56:13.411045 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:13.411049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:13.411117 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:13.411201 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:13.413278 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.414229 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.415597 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.416049 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.416143 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.420796 (+ 4653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:13.420806 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:13.420820 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:13.420847 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:13.424139 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:13.424781 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:13.424787 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:13.424790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:13.424941 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:13.425387 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:13.425389 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:13.425391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:13.425735 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:13.425979 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:13.426286 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 12:56:13.426539 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 12:56:13.434199 23479 tablet.cc:2369] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 4263e9adaebd4eaab9f38e2f0aed48f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:56:13.445367 23638 tablet.cc:2369] T d1f4d0ee3f4d4378b9424a6fa4fcb31e P 656d873850e54e4fbe5441651a43e253: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 12:56:13 dist-test-slave-gp6n krb5kdc[23052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280173, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 12:56:13 dist-test-slave-gp6n krb5kdc[23052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280173, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 12:56:13.484783 23652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.470014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50952 (local address 127.21.235.65:38271)
0809 12:56:13.470263 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:13.470271 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:13.470732 (+ 461us) server_negotiation.cc:407] Connection header received
0809 12:56:13.470973 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:13.470977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:13.471043 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:13.471150 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:13.472491 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.473299 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.474664 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.475115 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.475255 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.479365 (+ 4110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:13.479370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:13.479373 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:13.479406 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:13.481970 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:13.482717 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:13.482721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:13.482724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:13.482853 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:13.483275 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:13.483285 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:13.483287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:13.483553 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:13.483900 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:13.484131 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 12:56:13.484355 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
Aug 09 12:56:13 dist-test-slave-gp6n krb5kdc[23052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 12:56:13 dist-test-slave-gp6n krb5kdc[23052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 12:56:13.535228 23652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.519098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50958 (local address 127.21.235.65:38271)
0809 12:56:13.519312 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:13.519316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:13.519344 (+ 28us) server_negotiation.cc:407] Connection header received
0809 12:56:13.519522 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:13.519527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:13.519606 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:13.519694 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:13.521218 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.522282 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.523639 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.524067 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.524165 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.529046 (+ 4881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:13.529052 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:13.529058 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:13.529097 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:13.532420 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:13.533157 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:13.533163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:13.533167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:13.533329 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:13.533747 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:13.533752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:13.533755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:13.534062 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:13.534337 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:13.534656 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 12:56:13.534869 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260809 12:56:13.537837 23218 tablet_service.cc:2908] Rejecting scan request for tablet d1f4d0ee3f4d4378b9424a6fa4fcb31e: Uninitialized: clean time has not yet been initialized
I20260809 12:56:13.542213 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 23168
I20260809 12:56:13.554682 23652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.543089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50972 (local address 127.21.235.65:38271)
0809 12:56:13.543253 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:13.543259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:13.543651 (+ 392us) server_negotiation.cc:407] Connection header received
0809 12:56:13.543772 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:13.543777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:13.543847 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:13.543935 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:13.545083 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.546130 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.547349 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.547732 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.547827 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.549887 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:13.549891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:13.549894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:13.549920 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:13.552282 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:13.552877 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:13.552881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:13.552883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:13.552992 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:13.553406 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:13.553410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:13.553412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:13.553645 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:13.553822 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:13.554092 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 12:56:13.554272 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 12:56:13.555414 23311 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:13.649026 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 23168
I20260809 12:56:13.668285 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 23326
Aug 09 12:56:13 dist-test-slave-gp6n krb5kdc[23052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.66@KRBTEST.COM
I20260809 12:56:13.683769 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.669100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42490 (local address 127.21.235.66:37761)
0809 12:56:13.669292 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:13.669297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:13.669314 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:56:13.669450 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:13.669453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:13.669524 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:13.669606 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:13.670935 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.671757 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.673027 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.673438 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.673547 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.677624 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:13.677636 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:13.677651 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:13.677682 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:13.680749 (+ 3067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:13.681535 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:13.681542 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:13.681544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:13.681669 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:13.682129 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:13.682133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:13.682136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:13.682632 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:13.682916 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:13.683233 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 12:56:13.683412 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 12:56:13.684478 23473 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:13.772833 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 23326
I20260809 12:56:13.792943 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 23485
Aug 09 12:56:13 dist-test-slave-gp6n krb5kdc[23052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.67@KRBTEST.COM
I20260809 12:56:13.809837 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:13.793778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49354 (local address 127.21.235.67:45427)
0809 12:56:13.793951 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:13.793955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:13.793973 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:56:13.794036 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:13.794040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:13.794101 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:13.794188 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:13.795540 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.796391 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.797731 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:13.798174 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:13.798287 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:13.803397 (+ 5110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:13.803413 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:13.803429 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:13.803472 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:13.806770 (+ 3298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:13.807535 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:13.807544 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:13.807546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:13.807706 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:13.808144 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:13.808151 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:13.808155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:13.808550 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:13.808896 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:13.809206 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 12:56:13.809471 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 12:56:13.810571 23631 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:13.899529 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 23485
I20260809 12:56:13.919749 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 23066
I20260809 12:56:13.921052 23136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:14.031522 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 23066
2026-08-09T12:56:14Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (5569 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:56:14 dist-test-slave-gp6n krb5kdc[23686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:56:14 dist-test-slave-gp6n krb5kdc[23686](info): set up 2 sockets
Aug 09 12:56:14 dist-test-slave-gp6n krb5kdc[23686](info): commencing operation
krb5kdc: starting...
W20260809 12:56:15.196728 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:56:15 dist-test-slave-gp6n krb5kdc[23686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:56:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:56:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 12:56:15.398064 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:45333
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:45333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:15.723192 23700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:15.723537 23700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:15.723694 23700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:15.730931 23700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:56:15.731082 23700 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 12:56:15.731159 23700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:15.731261 23700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:15.731346 23700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:56:15.731436 23700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:56:15.741253 23700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:45333
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:45333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:15.742637 23700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:15.743992 23700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:15.756449 23709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:15.756408 23707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:15.756527 23706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:15.757619 23700 server_base.cc:1034] running on GCE node
I20260809 12:56:15.758374 23700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:15.760166 23700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:15.761421 23700 hybrid_clock.cc:648] HybridClock initialized: now 1786280175761360 us; error 45 us; skew 500 ppm
I20260809 12:56:15.761811 23700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:15 dist-test-slave-gp6n krb5kdc[23686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280175, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:15.765086 23700 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:56:15.766884 23700 webserver.cc:460] Webserver started at http://127.21.235.126:41269/ using document root <none> and password file <none>
I20260809 12:56:15.767624 23700 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:15.767716 23700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:15.768057 23700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:15.771154 23700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "45c1b63f18934868b023be2d389b53e8"
format_stamp: "Formatted at 2026-08-09 12:56:15 on dist-test-slave-gp6n"
server_key: "294300248ff8270befc55ef9ad49800f"
server_key_iv: "b2b403499bb21704f1ed1990ac18a16b"
server_key_version: "encryptionkey@0"
I20260809 12:56:15.771970 23700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "45c1b63f18934868b023be2d389b53e8"
format_stamp: "Formatted at 2026-08-09 12:56:15 on dist-test-slave-gp6n"
server_key: "294300248ff8270befc55ef9ad49800f"
server_key_iv: "b2b403499bb21704f1ed1990ac18a16b"
server_key_version: "encryptionkey@0"
I20260809 12:56:15.777487 23700 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:56:15.781196 23716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:15.782708 23700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 12:56:15.782903 23700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "45c1b63f18934868b023be2d389b53e8"
format_stamp: "Formatted at 2026-08-09 12:56:15 on dist-test-slave-gp6n"
server_key: "294300248ff8270befc55ef9ad49800f"
server_key_iv: "b2b403499bb21704f1ed1990ac18a16b"
server_key_version: "encryptionkey@0"
I20260809 12:56:15.783145 23700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:15.795126 23700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:15.799423 23700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:15.799690 23700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:15.822093 23700 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:45333
I20260809 12:56:15.822111 23786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:45333 every 8 connection(s)
I20260809 12:56:15.823557 23700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:56:15.826979 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 23700
I20260809 12:56:15.827221 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:56:15.827986 23787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:15.828096 22445 external_mini_cluster.cc:1428] Setting key 03692a0ea5d20d21c5ef74d38763aa25
I20260809 12:56:15.843017 23787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8: Bootstrap starting.
Aug 09 12:56:15 dist-test-slave-gp6n krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:15.847108 23787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:15.848497 23787 log.cc:824] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:15.852187 23787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8: No bootstrap required, opened a new log
I20260809 12:56:15.853556 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:15.830270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34666 (local address 127.21.235.126:45333)
0809 12:56:15.831169 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:15.831180 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:15.831224 (+ 44us) server_negotiation.cc:407] Connection header received
0809 12:56:15.832974 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:15.832995 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:15.833386 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:15.833709 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:15.835510 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:15.836815 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:15.838596 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:15.839299 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:15.839431 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:15.844310 (+ 4879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:15.844321 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:15.844341 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:15.844403 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:15.848655 (+ 4252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:15.849577 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:15.849583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:15.849589 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:15.849863 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:15.850291 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:15.850317 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:15.850320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:15.851073 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:15.851392 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:15.851729 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 12:56:15.852194 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":453,"threads_started":1}
I20260809 12:56:15.860658 23787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c1b63f18934868b023be2d389b53e8" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 45333 } }
I20260809 12:56:15.860977 23787 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:15.861049 23787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c1b63f18934868b023be2d389b53e8, State: Initialized, Role: FOLLOWER
I20260809 12:56:15.861519 23787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c1b63f18934868b023be2d389b53e8" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 45333 } }
I20260809 12:56:15.861825 23787 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:56:15.861964 23787 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:56:15.862087 23787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:15.864291 23787 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c1b63f18934868b023be2d389b53e8" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 45333 } }
I20260809 12:56:15.864801 23787 leader_election.cc:302] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45c1b63f18934868b023be2d389b53e8; no voters:
I20260809 12:56:15.865355 23787 leader_election.cc:288] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:56:15.865504 23792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:15.866212 23792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8 [term 1 LEADER]: Becoming Leader. State: Replica: 45c1b63f18934868b023be2d389b53e8, State: Running, Role: LEADER
I20260809 12:56:15.866760 23792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c1b63f18934868b023be2d389b53e8" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 45333 } }
I20260809 12:56:15.867422 23787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:56:15.869808 23793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45c1b63f18934868b023be2d389b53e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c1b63f18934868b023be2d389b53e8" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 45333 } } }
I20260809 12:56:15.870299 23793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8 [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:15.870174 23795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45c1b63f18934868b023be2d389b53e8. Latest consensus state: current_term: 1 leader_uuid: "45c1b63f18934868b023be2d389b53e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c1b63f18934868b023be2d389b53e8" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 45333 } } }
I20260809 12:56:15.871039 23795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8 [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:15.872236 23798 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:56:15.878523 23798 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:56:15.888404 23798 catalog_manager.cc:1347] Generated new cluster ID: b62a6f1bcb1b4436911328c63b5a482a
I20260809 12:56:15.888525 23798 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:56:15.906836 23798 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:56:15.908360 23798 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:56:15.916956 23798 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8: Generated new TSK 0
I20260809 12:56:15.917943 23798 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
I20260809 12:56:15.978252 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:45333
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:16.317727 23816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:16.318034 23816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:16.318213 23816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:16.325320 23816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:16.325474 23816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:16.325628 23816 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:56:16.325786 23816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:56:16.336223 23816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.235.126:45333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:16.337496 23816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:16.338901 23816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:16.351773 23826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:16.351878 23824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:16.351944 23823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:16.352936 23816 server_base.cc:1034] running on GCE node
I20260809 12:56:16.353533 23816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:16.354532 23816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:16.355893 23816 hybrid_clock.cc:648] HybridClock initialized: now 1786280176355813 us; error 69 us; skew 500 ppm
I20260809 12:56:16.356222 23816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:16 dist-test-slave-gp6n krb5kdc[23686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280176, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:16.359401 23816 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 12:56:16.360870 23816 webserver.cc:460] Webserver started at http://127.21.235.65:36935/ using document root <none> and password file <none>
I20260809 12:56:16.361594 23816 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:16.361729 23816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:16.362061 23816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:16.365121 23816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "11da07ecf753480ba403b9318fa2eb27"
format_stamp: "Formatted at 2026-08-09 12:56:16 on dist-test-slave-gp6n"
server_key: "93f5a950ccbd83063ff22c4bddf22b16"
server_key_iv: "13e15d2213c7f4c97dd8c7474a6dc15f"
server_key_version: "encryptionkey@0"
I20260809 12:56:16.365935 23816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "11da07ecf753480ba403b9318fa2eb27"
format_stamp: "Formatted at 2026-08-09 12:56:16 on dist-test-slave-gp6n"
server_key: "93f5a950ccbd83063ff22c4bddf22b16"
server_key_iv: "13e15d2213c7f4c97dd8c7474a6dc15f"
server_key_version: "encryptionkey@0"
I20260809 12:56:16.371250 23816 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 12:56:16.374780 23834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:16.376101 23816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:56:16.376282 23816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "11da07ecf753480ba403b9318fa2eb27"
format_stamp: "Formatted at 2026-08-09 12:56:16 on dist-test-slave-gp6n"
server_key: "93f5a950ccbd83063ff22c4bddf22b16"
server_key_iv: "13e15d2213c7f4c97dd8c7474a6dc15f"
server_key_version: "encryptionkey@0"
I20260809 12:56:16.376525 23816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:16.407435 23816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:16.411736 23816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:16.412019 23816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:16.414878 23816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:16.414997 23816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:16.415174 23816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:16.415261 23816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:56:16 dist-test-slave-gp6n krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280176, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:16.439342 23844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:16.415466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:44915 (local address 127.21.235.126:45333)
0809 12:56:16.416455 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:16.416464 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:16.417257 (+ 793us) server_negotiation.cc:407] Connection header received
0809 12:56:16.419614 (+ 2357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:16.419620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:16.419707 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:16.419836 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:16.422063 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:16.422972 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:16.425636 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:16.426243 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:16.426386 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:16.431161 (+ 4775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:16.431172 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:16.431180 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:16.431236 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:16.435466 (+ 4230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:16.436298 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:16.436303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:16.436309 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:16.436493 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:16.437183 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:16.437189 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:16.437192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:16.437612 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:16.437807 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:16.438747 (+ 940us) server_negotiation.cc:299] Negotiation successful
0809 12:56:16.438975 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":839,"thread_start_us":738,"threads_started":1}
I20260809 12:56:16.440361 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:16.416164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:45333 (local address 127.21.235.65:44915)
0809 12:56:16.417099 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:16.417157 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:16.419349 (+ 2192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:16.420132 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:16.420149 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:16.421001 (+ 852us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:16.421889 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:16.421909 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:16.423178 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:16.423181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:16.425478 (+ 2297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:16.425495 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:16.426568 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:16.426576 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:16.426811 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:16.427555 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:16.427595 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:16.430885 (+ 3290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:16.435631 (+ 4746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:16.435646 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:16.435664 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:16.436153 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:16.436631 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:16.436638 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:16.436644 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:16.436984 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:16.437787 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:16.437800 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:16.438306 (+ 506us) client_negotiation.cc:769] Sending connection context
0809 12:56:16.438650 (+ 344us) client_negotiation.cc:240] Negotiation successful
0809 12:56:16.439198 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":426,"threads_started":1}
I20260809 12:56:16.461205 23844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:16.446731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:53185 (local address 127.21.235.126:45333)
0809 12:56:16.446915 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:16.446922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:16.447737 (+ 815us) server_negotiation.cc:407] Connection header received
0809 12:56:16.447963 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:16.447970 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:16.448053 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:16.448180 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:16.449156 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:16.449888 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:16.451608 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:16.452025 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:16.452178 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:16.454352 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:16.454361 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:16.454367 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:16.454444 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:16.458301 (+ 3857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:16.459051 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:16.459059 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:16.459065 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:16.459231 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:16.459754 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:16.459757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:16.459760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:16.460023 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:16.460193 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:16.460608 (+ 415us) server_negotiation.cc:299] Negotiation successful
0809 12:56:16.460834 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:56:16.461344 23909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:16.446829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:45333 (local address 127.21.235.65:53185)
0809 12:56:16.447602 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:16.447632 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:16.447801 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:16.448206 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:16.448211 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:16.448716 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:16.449005 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:16.449014 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:16.450024 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:16.450029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:16.451451 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:16.451466 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:16.452221 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:16.452227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:16.452400 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:16.453085 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:16.453100 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:16.454087 (+ 987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:16.458492 (+ 4405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:16.458496 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:16.458499 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:16.458925 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:16.459387 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:16.459397 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:16.459403 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:16.459591 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:16.460148 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:16.460153 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:16.460344 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 12:56:16.460506 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 12:56:16.460840 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":383,"threads_started":1}
I20260809 12:56:16.471897 23816 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:36187
I20260809 12:56:16.471948 23971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:36187 every 8 connection(s)
I20260809 12:56:16.473122 23816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:56:16.475176 23972 heartbeater.cc:347] Connected to a master server at 127.21.235.126:45333
I20260809 12:56:16.475406 23972 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:16.476033 23972 heartbeater.cc:511] Master 127.21.235.126:45333 requested a full tablet report, sending...
I20260809 12:56:16.478106 23737 ts_manager.cc:194] Registered new tserver with Master: 11da07ecf753480ba403b9318fa2eb27 (127.21.235.65:36187)
I20260809 12:56:16.479104 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 23816
I20260809 12:56:16.479326 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:56:16.479740 23737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:44915
I20260809 12:56:16.480088 22445 external_mini_cluster.cc:1428] Setting key b9df837ae697a92c15d80661f7d8013c
WARNING: no policy specified for kudu/127.21.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
I20260809 12:56:16.521533 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:45333
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:16.841403 23977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:16.841674 23977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:16.841773 23977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:16.849088 23977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:16.849198 23977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:16.849300 23977 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:56:16.849390 23977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:56:16.859191 23977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.235.126:45333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:16.860384 23977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:16.861994 23977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:16.876251 23984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:16.876327 23985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:16.876518 23987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:16.877427 23977 server_base.cc:1034] running on GCE node
I20260809 12:56:16.878105 23977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:16.879115 23977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:16.880343 23977 hybrid_clock.cc:648] HybridClock initialized: now 1786280176880279 us; error 36 us; skew 500 ppm
I20260809 12:56:16.880650 23977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:16 dist-test-slave-gp6n krb5kdc[23686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280176, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:16.889214 23977 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 12:56:16.890861 23977 webserver.cc:460] Webserver started at http://127.21.235.66:38115/ using document root <none> and password file <none>
I20260809 12:56:16.891535 23977 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:16.891624 23977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:16.891995 23977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:16.894990 23977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "3ecc3d3f697b4402932082cd23fd42ee"
format_stamp: "Formatted at 2026-08-09 12:56:16 on dist-test-slave-gp6n"
server_key: "b7fd92f18e17e432bdedd25ab122b83b"
server_key_iv: "53d8fff2949386459148ca4fdbb7bd50"
server_key_version: "encryptionkey@0"
I20260809 12:56:16.895746 23977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "3ecc3d3f697b4402932082cd23fd42ee"
format_stamp: "Formatted at 2026-08-09 12:56:16 on dist-test-slave-gp6n"
server_key: "b7fd92f18e17e432bdedd25ab122b83b"
server_key_iv: "53d8fff2949386459148ca4fdbb7bd50"
server_key_version: "encryptionkey@0"
I20260809 12:56:16.900877 23977 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 12:56:16.904322 23995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:16.905727 23977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:56:16.905936 23977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "3ecc3d3f697b4402932082cd23fd42ee"
format_stamp: "Formatted at 2026-08-09 12:56:16 on dist-test-slave-gp6n"
server_key: "b7fd92f18e17e432bdedd25ab122b83b"
server_key_iv: "53d8fff2949386459148ca4fdbb7bd50"
server_key_version: "encryptionkey@0"
I20260809 12:56:16.906178 23977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:16.935200 23977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:16.940207 23977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:16.940546 23977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:16.943997 23977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:16.944128 23977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:16.944353 23977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:16.944437 23977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:56:16 dist-test-slave-gp6n krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280176, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:16.971040 23844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:16.944242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:45737 (local address 127.21.235.126:45333)
0809 12:56:16.944494 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:16.944501 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:16.946153 (+ 1652us) server_negotiation.cc:407] Connection header received
0809 12:56:16.949543 (+ 3390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:16.949550 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:16.949649 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:16.949762 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:16.952082 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:16.953168 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:16.955982 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:16.956630 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:16.956773 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:16.962125 (+ 5352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:16.962131 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:16.962134 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:16.962169 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:16.966772 (+ 4603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:16.968082 (+ 1310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:16.968090 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:16.968097 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:16.968271 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:16.968803 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:16.968807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:16.968809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:16.969096 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:16.969227 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:16.970586 (+ 1359us) server_negotiation.cc:299] Negotiation successful
0809 12:56:16.970768 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 12:56:16.973439 24005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:16.944937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:45333 (local address 127.21.235.66:45737)
0809 12:56:16.945969 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:16.946022 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:16.949121 (+ 3099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:16.950126 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:16.950136 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:16.950895 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:16.951823 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:16.951855 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:16.953401 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:16.953408 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:16.955812 (+ 2404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:16.955830 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:16.956895 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:16.956903 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:16.957129 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:16.957944 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:16.957992 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:16.961873 (+ 3881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:16.967048 (+ 5175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:16.967063 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:16.967086 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:16.967882 (+ 796us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:16.968425 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:16.968428 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:16.968430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:16.968642 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:16.969252 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:16.969272 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:16.969957 (+ 685us) client_negotiation.cc:769] Sending connection context
0809 12:56:16.970484 (+ 527us) client_negotiation.cc:240] Negotiation successful
0809 12:56:16.971305 (+ 821us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":350,"threads_started":1}
I20260809 12:56:17.002393 23844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:16.985712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:49199 (local address 127.21.235.126:45333)
0809 12:56:16.985962 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:16.985971 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:16.986803 (+ 832us) server_negotiation.cc:407] Connection header received
0809 12:56:16.987073 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:16.987110 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:16.987193 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:16.987279 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:16.988945 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:16.989683 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:16.992427 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:16.992855 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:16.992966 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:16.996001 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:16.996006 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:16.996009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:16.996042 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:16.999072 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:16.999806 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:16.999811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:16.999814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:16.999947 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:17.000438 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:17.000445 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:17.000448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:17.001050 (+ 602us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:17.001261 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:17.001792 (+ 531us) server_negotiation.cc:299] Negotiation successful
0809 12:56:17.002035 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 12:56:17.002611 24089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:16.985796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:45333 (local address 127.21.235.66:49199)
0809 12:56:16.986662 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:16.986692 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:16.986936 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:16.987355 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:16.987364 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:16.988221 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:16.988759 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:16.988780 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:16.989862 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:16.989871 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:16.992242 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:16.992264 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:16.993087 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:16.993102 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:16.993389 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:16.994388 (+ 999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:16.994455 (+ 67us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:16.995781 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:16.999207 (+ 3426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:16.999213 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:16.999217 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:16.999667 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:17.000074 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:17.000083 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:17.000086 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:17.000263 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:17.001219 (+ 956us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:17.001226 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:17.001488 (+ 262us) client_negotiation.cc:769] Sending connection context
0809 12:56:17.001675 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 12:56:17.002068 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":400,"threads_started":1}
I20260809 12:56:17.012032 23977 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:38077
I20260809 12:56:17.012079 24133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:38077 every 8 connection(s)
I20260809 12:56:17.013387 23977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:56:17.015632 24134 heartbeater.cc:347] Connected to a master server at 127.21.235.126:45333
I20260809 12:56:17.015873 24134 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:17.016563 24134 heartbeater.cc:511] Master 127.21.235.126:45333 requested a full tablet report, sending...
I20260809 12:56:17.018016 23737 ts_manager.cc:194] Registered new tserver with Master: 3ecc3d3f697b4402932082cd23fd42ee (127.21.235.66:38077)
I20260809 12:56:17.018929 23737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:45737
I20260809 12:56:17.019318 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 23977
I20260809 12:56:17.019542 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:56:17.020090 22445 external_mini_cluster.cc:1428] Setting key 9dd7b8dba43dce1897c7f8709b089211
WARNING: no policy specified for kudu/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
I20260809 12:56:17.066757 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:45333
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:17.398563 24138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:17.398886 24138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:17.399067 24138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:17.406194 24138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:17.406364 24138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:17.406533 24138 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:56:17.406677 24138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:56:17.416486 24138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.235.126:45333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:17.418280 24138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:17.419616 24138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:17.432317 24143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:17.432380 24144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:17.432411 24146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:17.433532 24138 server_base.cc:1034] running on GCE node
I20260809 12:56:17.434131 24138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:17.435118 24138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:17.436334 24138 hybrid_clock.cc:648] HybridClock initialized: now 1786280177436282 us; error 36 us; skew 500 ppm
I20260809 12:56:17.436594 24138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:17 dist-test-slave-gp6n krb5kdc[23686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280177, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:17.439812 24138 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 12:56:17.441159 24138 webserver.cc:460] Webserver started at http://127.21.235.67:37889/ using document root <none> and password file <none>
I20260809 12:56:17.441818 24138 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:17.441905 24138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:17.442163 24138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:17.445328 24138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "06d7790219984fd3b1650b4e25f7429c"
format_stamp: "Formatted at 2026-08-09 12:56:17 on dist-test-slave-gp6n"
server_key: "65c93df6defc6cc7f36ecac360ececad"
server_key_iv: "13665b1deb3cd42d4d38ce181f22eb18"
server_key_version: "encryptionkey@0"
I20260809 12:56:17.446097 24138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "06d7790219984fd3b1650b4e25f7429c"
format_stamp: "Formatted at 2026-08-09 12:56:17 on dist-test-slave-gp6n"
server_key: "65c93df6defc6cc7f36ecac360ececad"
server_key_iv: "13665b1deb3cd42d4d38ce181f22eb18"
server_key_version: "encryptionkey@0"
I20260809 12:56:17.451131 24138 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 12:56:17.454681 24153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:17.456044 24138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:56:17.456214 24138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "06d7790219984fd3b1650b4e25f7429c"
format_stamp: "Formatted at 2026-08-09 12:56:17 on dist-test-slave-gp6n"
server_key: "65c93df6defc6cc7f36ecac360ececad"
server_key_iv: "13665b1deb3cd42d4d38ce181f22eb18"
server_key_version: "encryptionkey@0"
I20260809 12:56:17.456413 24138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:17.470093 24138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:17.474663 24138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:17.474961 24138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:17.478190 24138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:17.478319 24138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:17.478495 24138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:17.478602 24138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:17.482821 23972 heartbeater.cc:503] Master 127.21.235.126:45333 was elected leader, sending a full tablet report...
Aug 09 12:56:17 dist-test-slave-gp6n krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280177, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:17.503014 23844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:17.478496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:59541 (local address 127.21.235.126:45333)
0809 12:56:17.478711 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:17.478718 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:17.480269 (+ 1551us) server_negotiation.cc:407] Connection header received
0809 12:56:17.482616 (+ 2347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:17.482622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:17.482708 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:17.482802 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:17.485679 (+ 2877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.486519 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.489386 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.489982 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.490119 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:17.495518 (+ 5399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:17.495524 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:17.495527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:17.495562 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:17.499282 (+ 3720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:17.500437 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:17.500443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:17.500446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:17.500596 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:17.501065 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:17.501070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:17.501074 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:17.501383 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:17.501545 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:17.502356 (+ 811us) server_negotiation.cc:299] Negotiation successful
0809 12:56:17.502629 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:56:17.504578 24163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:17.479148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:45333 (local address 127.21.235.67:59541)
0809 12:56:17.480109 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:17.480157 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:17.482294 (+ 2137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:17.483544 (+ 1250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:17.483556 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:17.484486 (+ 930us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:17.485500 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:17.485521 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.486736 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.486741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:17.489142 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:17.489163 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.490244 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.490252 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:17.490560 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:17.491535 (+ 975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:17.491562 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:17.495212 (+ 3650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:17.499450 (+ 4238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:17.499466 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:17.499491 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:17.500250 (+ 759us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:17.500716 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:17.500721 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:17.500728 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:17.500946 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:17.501527 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:17.501534 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:17.501926 (+ 392us) client_negotiation.cc:769] Sending connection context
0809 12:56:17.502226 (+ 300us) client_negotiation.cc:240] Negotiation successful
0809 12:56:17.502865 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":394,"threads_started":1}
I20260809 12:56:17.524178 23844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:17.510180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:45767 (local address 127.21.235.126:45333)
0809 12:56:17.510329 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:17.510333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:17.511092 (+ 759us) server_negotiation.cc:407] Connection header received
0809 12:56:17.511316 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:17.511321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:17.511406 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:17.511508 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:17.512519 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.513536 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.515619 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.516015 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.516148 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:17.517922 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:17.517927 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:17.517930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:17.517959 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:17.521149 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:17.522168 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:17.522176 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:17.522180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:17.522313 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:17.522759 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:17.522767 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:17.522771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:17.523025 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:17.523199 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:17.523579 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 12:56:17.523802 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 12:56:17.524266 24232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:17.510301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:45333 (local address 127.21.235.67:45767)
0809 12:56:17.510983 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:17.511003 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:17.511172 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:17.511590 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:17.511594 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:17.512101 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:17.512381 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:17.512389 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.513675 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.513690 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:17.515470 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:17.515483 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.516204 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.516210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:17.516369 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:17.517028 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:17.517044 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:17.517737 (+ 693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:17.521325 (+ 3588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:17.521330 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:17.521335 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:17.522034 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:17.522467 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:17.522472 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:17.522480 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:17.522625 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:17.523150 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:17.523154 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:17.523338 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 12:56:17.523476 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 12:56:17.523812 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":429,"threads_started":1}
I20260809 12:56:17.536976 24138 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:39805
I20260809 12:56:17.537047 24288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:39805 every 8 connection(s)
I20260809 12:56:17.538193 24138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:56:17.540271 24289 heartbeater.cc:347] Connected to a master server at 127.21.235.126:45333
I20260809 12:56:17.540503 24289 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:17.541154 24289 heartbeater.cc:511] Master 127.21.235.126:45333 requested a full tablet report, sending...
I20260809 12:56:17.542526 23737 ts_manager.cc:194] Registered new tserver with Master: 06d7790219984fd3b1650b4e25f7429c (127.21.235.67:39805)
I20260809 12:56:17.543376 23737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:59541
I20260809 12:56:17.546334 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 24138
I20260809 12:56:17.546566 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:56:17.547013 22445 external_mini_cluster.cc:1428] Setting key 4fe317dcf4d646edd944e0e94ac6c687
I20260809 12:56:17.549865 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:56:17 dist-test-slave-gp6n krb5kdc[23686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280177, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 12:56:17 dist-test-slave-gp6n krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280177, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:17.599713 23844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:17.586138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34672 (local address 127.21.235.126:45333)
0809 12:56:17.586319 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:17.586324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:17.586798 (+ 474us) server_negotiation.cc:407] Connection header received
0809 12:56:17.586989 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:17.586993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:17.587062 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:17.587157 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:17.588533 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.589290 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.590611 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.591028 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.591128 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:17.594915 (+ 3787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:17.594920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:17.594923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:17.594965 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:17.597350 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:17.597925 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:17.597928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:17.597931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:17.598057 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:17.598444 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:17.598450 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:17.598453 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:17.598733 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:17.598926 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:17.599276 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 12:56:17.599436 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:56:17.603732 23737 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:56:17.610720 23737 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:56:17.647440 24307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:17.623132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35216 (local address 127.21.235.67:39805)
0809 12:56:17.638029 (+ 14897us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:17.638039 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:17.638073 (+ 34us) server_negotiation.cc:407] Connection header received
0809 12:56:17.638179 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:17.638191 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:17.638512 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:17.638860 (+ 348us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:17.640156 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.641838 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.644762 (+ 2924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.646132 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.646256 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:17.646483 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:17.646943 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 12:56:17.647109 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14688,"thread_start_us":14478,"threads_started":1}
I20260809 12:56:17.647528 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:17.625375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:36187 (local address 127.0.0.1:40474)
0809 12:56:17.638291 (+ 12916us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:17.638312 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:17.638448 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:17.639482 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:17.639487 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:17.639513 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:17.640074 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:17.640087 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.642264 (+ 2177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.642273 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:17.644349 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:17.644361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.646291 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.646301 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:17.646610 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:17.646658 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 12:56:17.646798 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 12:56:17.647007 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12796,"mutex_wait_us":142,"thread_start_us":7494,"threads_started":1}
I20260809 12:56:17.648535 24309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:17.622796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:39805 (local address 127.0.0.1:35216)
0809 12:56:17.633439 (+ 10643us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:17.633490 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:17.633722 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:17.639220 (+ 5498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:17.639226 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:17.639253 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:17.639918 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:17.639931 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.642007 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.642012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:17.644595 (+ 2583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:17.644610 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.646335 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.646341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:17.646579 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:17.646657 (+ 78us) client_negotiation.cc:769] Sending connection context
0809 12:56:17.646798 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 12:56:17.648130 (+ 1332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10301,"mutex_wait_us":1195,"thread_start_us":15429,"threads_started":1}
I20260809 12:56:17.648674 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:17.626061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40474 (local address 127.21.235.65:36187)
0809 12:56:17.634467 (+ 8406us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:17.634480 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:17.638573 (+ 4093us) server_negotiation.cc:407] Connection header received
0809 12:56:17.638677 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:17.638686 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:17.638966 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:17.639250 (+ 284us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:17.640283 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.642129 (+ 1846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.644520 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.646008 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.646161 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:17.646433 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:17.648170 (+ 1737us) server_negotiation.cc:299] Negotiation successful
0809 12:56:17.648366 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8155,"thread_start_us":7989,"threads_started":1}
I20260809 12:56:17.649015 24305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:17.623135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:38077 (local address 127.0.0.1:39496)
0809 12:56:17.637762 (+ 14627us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:17.637783 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:17.638064 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:17.638978 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:17.638987 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:17.639029 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:17.639816 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:17.639837 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.641216 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.641221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:17.644595 (+ 3374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:17.644610 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.648131 (+ 3521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.648137 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:17.648331 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:17.648362 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 12:56:17.648483 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 12:56:17.648625 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14489,"mutex_wait_us":290,"thread_start_us":14891,"threads_started":1}
I20260809 12:56:17.649211 24308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:17.623406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39496 (local address 127.21.235.66:38077)
0809 12:56:17.637407 (+ 14001us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:17.637418 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:17.638229 (+ 811us) server_negotiation.cc:407] Connection header received
0809 12:56:17.638353 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:17.638364 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:17.638641 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:17.639554 (+ 913us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:17.640064 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.641066 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.646552 (+ 5486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:17.647879 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:17.648057 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:17.648266 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:17.648695 (+ 429us) server_negotiation.cc:299] Negotiation successful
0809 12:56:17.648874 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13796,"thread_start_us":13634,"threads_started":1}
I20260809 12:56:17.652499 23888 tablet_service.cc:1467] Processing CreateTablet for tablet 7121ad38412a4298948f4130079d806c (DEFAULT_TABLE table=test-table [id=1e9abcf52b7d494dac4ab67ae24a92c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:17.652536 24208 tablet_service.cc:1467] Processing CreateTablet for tablet 7121ad38412a4298948f4130079d806c (DEFAULT_TABLE table=test-table [id=1e9abcf52b7d494dac4ab67ae24a92c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:17.652536 24051 tablet_service.cc:1467] Processing CreateTablet for tablet 7121ad38412a4298948f4130079d806c (DEFAULT_TABLE table=test-table [id=1e9abcf52b7d494dac4ab67ae24a92c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:17.653795 23888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7121ad38412a4298948f4130079d806c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:17.653927 24208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7121ad38412a4298948f4130079d806c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:17.654031 24051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7121ad38412a4298948f4130079d806c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:17.665616 24311 tablet_bootstrap.cc:492] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27: Bootstrap starting.
I20260809 12:56:17.670101 24311 tablet_bootstrap.cc:654] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:17.671205 24312 tablet_bootstrap.cc:492] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c: Bootstrap starting.
I20260809 12:56:17.671669 24311 log.cc:824] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:17.672122 24314 tablet_bootstrap.cc:492] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee: Bootstrap starting.
I20260809 12:56:17.675100 24311 tablet_bootstrap.cc:492] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27: No bootstrap required, opened a new log
I20260809 12:56:17.675405 24311 ts_tablet_manager.cc:1397] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 12:56:17.677170 24314 tablet_bootstrap.cc:654] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:17.678608 24312 tablet_bootstrap.cc:654] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:17.678905 24314 log.cc:824] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:17.680066 24312 log.cc:824] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:17.685050 24311 raft_consensus.cc:348] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } }
I20260809 12:56:17.685451 24311 raft_consensus.cc:374] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:17.685560 24311 raft_consensus.cc:729] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11da07ecf753480ba403b9318fa2eb27, State: Initialized, Role: FOLLOWER
I20260809 12:56:17.686309 24311 consensus_queue.cc:260] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } }
I20260809 12:56:17.693830 24312 tablet_bootstrap.cc:492] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c: No bootstrap required, opened a new log
I20260809 12:56:17.694005 24311 ts_tablet_manager.cc:1428] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27: Time spent starting tablet: real 0.018s user 0.010s sys 0.007s
I20260809 12:56:17.694236 24312 ts_tablet_manager.cc:1397] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.010s
I20260809 12:56:17.694285 24314 tablet_bootstrap.cc:492] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee: No bootstrap required, opened a new log
I20260809 12:56:17.694615 24314 ts_tablet_manager.cc:1397] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee: Time spent bootstrapping tablet: real 0.023s user 0.010s sys 0.012s
I20260809 12:56:17.703192 24314 raft_consensus.cc:348] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } }
I20260809 12:56:17.703545 24314 raft_consensus.cc:374] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:17.703652 24314 raft_consensus.cc:729] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ecc3d3f697b4402932082cd23fd42ee, State: Initialized, Role: FOLLOWER
I20260809 12:56:17.704141 24312 raft_consensus.cc:348] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } }
I20260809 12:56:17.704314 24314 consensus_queue.cc:260] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } }
I20260809 12:56:17.704533 24312 raft_consensus.cc:374] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:17.704725 24312 raft_consensus.cc:729] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d7790219984fd3b1650b4e25f7429c, State: Initialized, Role: FOLLOWER
I20260809 12:56:17.705433 24134 heartbeater.cc:503] Master 127.21.235.126:45333 was elected leader, sending a full tablet report...
I20260809 12:56:17.705410 24312 consensus_queue.cc:260] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } }
I20260809 12:56:17.706095 24314 ts_tablet_manager.cc:1428] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260809 12:56:17.706969 24289 heartbeater.cc:503] Master 127.21.235.126:45333 was elected leader, sending a full tablet report...
I20260809 12:56:17.707581 24312 ts_tablet_manager.cc:1428] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
W20260809 12:56:17.725582 23973 tablet.cc:2369] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:56:17.765836 24135 tablet.cc:2369] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:56:17.789901 24290 tablet.cc:2369] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:56:17.996529 24321 raft_consensus.cc:482] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:56:17.996906 24321 raft_consensus.cc:504] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } }
I20260809 12:56:17.998822 24321 leader_election.cc:288] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ecc3d3f697b4402932082cd23fd42ee (127.21.235.66:38077), 11da07ecf753480ba403b9318fa2eb27 (127.21.235.65:36187)
I20260809 12:56:18.006119 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:17.999418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:56925 (local address 127.21.235.65:36187)
0809 12:56:17.999699 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:17.999707 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:17.999732 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:56:17.999854 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:17.999863 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:17.999968 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:18.000092 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:18.000633 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.001733 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.003995 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.005252 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.005428 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.005507 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:18.005703 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 12:56:18.005863 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260809 12:56:18.006081 24163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:17.999276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:36187 (local address 127.21.235.67:56925)
0809 12:56:17.999566 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:17.999592 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:17.999705 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:18.000141 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:18.000150 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:18.000182 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:18.000459 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:18.000475 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.001889 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.001892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:18.003785 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:18.003800 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.005403 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.005407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:18.005546 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.005571 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 12:56:18.005649 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 12:56:18.005741 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260809 12:56:18.007167 24308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:17.999306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:37533 (local address 127.21.235.66:38077)
0809 12:56:17.999518 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:17.999524 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:18.000422 (+ 898us) server_negotiation.cc:407] Connection header received
0809 12:56:18.000561 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:18.000569 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:18.000653 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:18.000763 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:18.001316 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.002369 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.004507 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.005803 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.006006 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.006074 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:18.006715 (+ 641us) server_negotiation.cc:299] Negotiation successful
0809 12:56:18.006879 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 12:56:18.007284 24322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:17.999484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:38077 (local address 127.21.235.67:37533)
0809 12:56:18.000160 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:18.000176 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:18.000404 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:18.000776 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:18.000779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:18.000799 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:18.001088 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:18.001095 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.002561 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.002566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:18.004315 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:18.004326 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.006221 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.006235 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:18.006538 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.006578 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 12:56:18.006678 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 12:56:18.006810 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":375,"threads_started":1}
I20260809 12:56:18.007428 23916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7121ad38412a4298948f4130079d806c" candidate_uuid: "06d7790219984fd3b1650b4e25f7429c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11da07ecf753480ba403b9318fa2eb27" is_pre_election: true
I20260809 12:56:18.008111 23916 raft_consensus.cc:2454] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d7790219984fd3b1650b4e25f7429c in term 0.
I20260809 12:56:18.008483 24072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7121ad38412a4298948f4130079d806c" candidate_uuid: "06d7790219984fd3b1650b4e25f7429c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ecc3d3f697b4402932082cd23fd42ee" is_pre_election: true
I20260809 12:56:18.008813 24157 leader_election.cc:302] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d7790219984fd3b1650b4e25f7429c, 11da07ecf753480ba403b9318fa2eb27; no voters:
I20260809 12:56:18.009138 24072 raft_consensus.cc:2454] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d7790219984fd3b1650b4e25f7429c in term 0.
I20260809 12:56:18.009323 24321 raft_consensus.cc:2781] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:56:18.009526 24321 raft_consensus.cc:482] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:56:18.009649 24321 raft_consensus.cc:3037] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:18.012749 24321 raft_consensus.cc:504] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } }
I20260809 12:56:18.013470 24321 leader_election.cc:288] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [CANDIDATE]: Term 1 election: Requested vote from peers 3ecc3d3f697b4402932082cd23fd42ee (127.21.235.66:38077), 11da07ecf753480ba403b9318fa2eb27 (127.21.235.65:36187)
I20260809 12:56:18.013983 24072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7121ad38412a4298948f4130079d806c" candidate_uuid: "06d7790219984fd3b1650b4e25f7429c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ecc3d3f697b4402932082cd23fd42ee"
I20260809 12:56:18.014112 23916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7121ad38412a4298948f4130079d806c" candidate_uuid: "06d7790219984fd3b1650b4e25f7429c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11da07ecf753480ba403b9318fa2eb27"
I20260809 12:56:18.014364 23916 raft_consensus.cc:3037] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:18.014386 24072 raft_consensus.cc:3037] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:18.017987 23916 raft_consensus.cc:2454] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d7790219984fd3b1650b4e25f7429c in term 1.
I20260809 12:56:18.018000 24072 raft_consensus.cc:2454] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d7790219984fd3b1650b4e25f7429c in term 1.
I20260809 12:56:18.018426 24157 leader_election.cc:302] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d7790219984fd3b1650b4e25f7429c, 11da07ecf753480ba403b9318fa2eb27; no voters:
I20260809 12:56:18.018731 24321 raft_consensus.cc:2781] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:18.019924 24321 raft_consensus.cc:686] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [term 1 LEADER]: Becoming Leader. State: Replica: 06d7790219984fd3b1650b4e25f7429c, State: Running, Role: LEADER
I20260809 12:56:18.020433 24321 consensus_queue.cc:237] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } }
I20260809 12:56:18.025063 23737 catalog_manager.cc:5495] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c reported cstate change: term changed from 0 to 1, leader changed from <none> to 06d7790219984fd3b1650b4e25f7429c (127.21.235.67). New cstate: current_term: 1 leader_uuid: "06d7790219984fd3b1650b4e25f7429c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } health_report { overall_health: UNKNOWN } } }
I20260809 12:56:18.120314 24307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:18.112772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35228 (local address 127.21.235.67:39805)
0809 12:56:18.113023 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:18.113034 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:18.113516 (+ 482us) server_negotiation.cc:407] Connection header received
0809 12:56:18.113807 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:18.113811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:18.113907 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:18.114030 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:56:18.114853 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.115548 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.117752 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.118280 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.118488 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.119028 (+ 540us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:56:18.119672 (+ 644us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:56:18.119831 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:18.119952 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 12:56:18.120037 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 12:56:18.131613 23916 raft_consensus.cc:1261] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27 [term 1 FOLLOWER]: Refusing update from remote peer 06d7790219984fd3b1650b4e25f7429c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:18.132977 24323 consensus_queue.cc:1035] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [LEADER]: Connected to new peer: Peer: permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:56:18.133622 24072 raft_consensus.cc:1261] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee [term 1 FOLLOWER]: Refusing update from remote peer 06d7790219984fd3b1650b4e25f7429c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:18.134572 24323 consensus_queue.cc:1035] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:56:18.171489 23737 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:56:18.171864 23737 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:56:18.175809 23737 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8: Sending DeleteTablet for 3 replicas of tablet 7121ad38412a4298948f4130079d806c
I20260809 12:56:18.177291 24051 tablet_service.cc:1514] Processing DeleteTablet for tablet 7121ad38412a4298948f4130079d806c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:18 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:39496
I20260809 12:56:18.177261 24208 tablet_service.cc:1514] Processing DeleteTablet for tablet 7121ad38412a4298948f4130079d806c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:18 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:35216
I20260809 12:56:18.177786 23888 tablet_service.cc:1514] Processing DeleteTablet for tablet 7121ad38412a4298948f4130079d806c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:18 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:40474
I20260809 12:56:18.184392 23737 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:56:18.185824 23737 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:56:18.195304 24339 tablet_replica.cc:330] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c: stopping tablet replica
I20260809 12:56:18.196130 24339 raft_consensus.cc:2229] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:56:18.197671 23887 tablet_service.cc:1467] Processing CreateTablet for tablet 3061b4dec07d41f9a2e94fe500ed4cf1 (DEFAULT_TABLE table=test-table [id=50a47e2e48b14407aa7e97611a0c8636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:18.197938 24050 tablet_service.cc:1467] Processing CreateTablet for tablet 3061b4dec07d41f9a2e94fe500ed4cf1 (DEFAULT_TABLE table=test-table [id=50a47e2e48b14407aa7e97611a0c8636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:18.198257 23887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3061b4dec07d41f9a2e94fe500ed4cf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:18.198568 24050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3061b4dec07d41f9a2e94fe500ed4cf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:18.198776 24338 tablet_replica.cc:330] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27: stopping tablet replica
I20260809 12:56:18.197487 24208 tablet_service.cc:1467] Processing CreateTablet for tablet 3061b4dec07d41f9a2e94fe500ed4cf1 (DEFAULT_TABLE table=test-table [id=50a47e2e48b14407aa7e97611a0c8636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:18.199280 24208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3061b4dec07d41f9a2e94fe500ed4cf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:18.199215 24338 raft_consensus.cc:2229] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:56:18.199784 24338 raft_consensus.cc:2258] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:18.204598 24314 tablet_bootstrap.cc:492] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee: Bootstrap starting.
I20260809 12:56:18.206075 24339 raft_consensus.cc:2258] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:18.206308 24338 ts_tablet_manager.cc:1905] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:18.207450 24314 tablet_bootstrap.cc:654] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:18.209514 24339 ts_tablet_manager.cc:1905] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:18.211495 24337 tablet_replica.cc:330] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee: stopping tablet replica
I20260809 12:56:18.211982 24337 raft_consensus.cc:2229] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:56:18.212491 24337 raft_consensus.cc:2258] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:18.215587 24337 ts_tablet_manager.cc:1905] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:18.218801 24338 ts_tablet_manager.cc:1918] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:56:18.219182 24338 log.cc:1196] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/7121ad38412a4298948f4130079d806c
I20260809 12:56:18.220295 24338 ts_tablet_manager.cc:1939] T 7121ad38412a4298948f4130079d806c P 11da07ecf753480ba403b9318fa2eb27: Deleting consensus metadata
I20260809 12:56:18.222808 23721 catalog_manager.cc:4841] TS 11da07ecf753480ba403b9318fa2eb27 (127.21.235.65:36187): tablet 7121ad38412a4298948f4130079d806c (table test-table [id=1e9abcf52b7d494dac4ab67ae24a92c9]) successfully deleted
I20260809 12:56:18.222929 24314 tablet_bootstrap.cc:492] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee: No bootstrap required, opened a new log
I20260809 12:56:18.223081 24314 ts_tablet_manager.cc:1397] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.005s
I20260809 12:56:18.223913 24314 raft_consensus.cc:348] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } }
I20260809 12:56:18.224185 24314 raft_consensus.cc:374] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:18.224364 24314 raft_consensus.cc:729] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ecc3d3f697b4402932082cd23fd42ee, State: Initialized, Role: FOLLOWER
I20260809 12:56:18.224723 24314 consensus_queue.cc:260] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } }
I20260809 12:56:18.227259 24314 ts_tablet_manager.cc:1428] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 12:56:18.227897 24337 ts_tablet_manager.cc:1918] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:56:18.228139 24337 log.cc:1196] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/7121ad38412a4298948f4130079d806c
I20260809 12:56:18.229074 24342 tablet_bootstrap.cc:492] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c: Bootstrap starting.
I20260809 12:56:18.229502 24337 ts_tablet_manager.cc:1939] T 7121ad38412a4298948f4130079d806c P 3ecc3d3f697b4402932082cd23fd42ee: Deleting consensus metadata
I20260809 12:56:18.231500 23719 catalog_manager.cc:4841] TS 3ecc3d3f697b4402932082cd23fd42ee (127.21.235.66:38077): tablet 7121ad38412a4298948f4130079d806c (table test-table [id=1e9abcf52b7d494dac4ab67ae24a92c9]) successfully deleted
I20260809 12:56:18.232265 24342 tablet_bootstrap.cc:654] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:18.232764 24340 tablet_bootstrap.cc:492] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27: Bootstrap starting.
I20260809 12:56:18.234122 24340 tablet_bootstrap.cc:654] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:18.234787 24339 ts_tablet_manager.cc:1918] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:56:18.235110 24339 log.cc:1196] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/7121ad38412a4298948f4130079d806c
I20260809 12:56:18.235960 24339 ts_tablet_manager.cc:1939] T 7121ad38412a4298948f4130079d806c P 06d7790219984fd3b1650b4e25f7429c: Deleting consensus metadata
I20260809 12:56:18.238451 23718 catalog_manager.cc:4841] TS 06d7790219984fd3b1650b4e25f7429c (127.21.235.67:39805): tablet 7121ad38412a4298948f4130079d806c (table test-table [id=1e9abcf52b7d494dac4ab67ae24a92c9]) successfully deleted
I20260809 12:56:18.240281 24340 tablet_bootstrap.cc:492] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27: No bootstrap required, opened a new log
I20260809 12:56:18.240463 24340 ts_tablet_manager.cc:1397] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260809 12:56:18.240932 24342 tablet_bootstrap.cc:492] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c: No bootstrap required, opened a new log
I20260809 12:56:18.241117 24342 ts_tablet_manager.cc:1397] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260809 12:56:18.241381 24340 raft_consensus.cc:348] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } }
I20260809 12:56:18.241662 24340 raft_consensus.cc:374] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:18.241775 24340 raft_consensus.cc:729] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11da07ecf753480ba403b9318fa2eb27, State: Initialized, Role: FOLLOWER
I20260809 12:56:18.242079 24342 raft_consensus.cc:348] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } }
I20260809 12:56:18.242360 24342 raft_consensus.cc:374] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:18.242532 24342 raft_consensus.cc:729] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d7790219984fd3b1650b4e25f7429c, State: Initialized, Role: FOLLOWER
I20260809 12:56:18.242318 24340 consensus_queue.cc:260] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } }
I20260809 12:56:18.242837 24342 consensus_queue.cc:260] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } }
I20260809 12:56:18.243534 24342 ts_tablet_manager.cc:1428] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 12:56:18.244356 24340 ts_tablet_manager.cc:1428] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 12:56:18.379280 24320 raft_consensus.cc:482] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:56:18.379590 24320 raft_consensus.cc:504] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } }
I20260809 12:56:18.381588 24320 leader_election.cc:288] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11da07ecf753480ba403b9318fa2eb27 (127.21.235.65:36187), 06d7790219984fd3b1650b4e25f7429c (127.21.235.67:39805)
I20260809 12:56:18.390030 24307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:18.382378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:34055 (local address 127.21.235.67:39805)
0809 12:56:18.382602 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:18.382608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:18.383105 (+ 497us) server_negotiation.cc:407] Connection header received
0809 12:56:18.383281 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:18.383288 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:18.383402 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:18.383498 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:18.384426 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.385520 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.387755 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.388843 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.389008 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.389081 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:18.389540 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 12:56:18.389679 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260809 12:56:18.390120 24347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:18.381992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:39805 (local address 127.21.235.66:34055)
0809 12:56:18.382992 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:18.383008 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:18.383148 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:18.383726 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:18.383731 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:18.383758 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:18.384256 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:18.384269 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.385647 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.385653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:18.387598 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:18.387611 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.389141 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.389148 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:18.389349 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.389385 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 12:56:18.389488 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 12:56:18.389630 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":858,"mutex_wait_us":31,"thread_start_us":1428,"threads_started":1}
I20260809 12:56:18.390946 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:18.382805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:57071 (local address 127.21.235.65:36187)
0809 12:56:18.383009 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:18.383015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:18.383046 (+ 31us) server_negotiation.cc:407] Connection header received
0809 12:56:18.383133 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:18.383138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:18.383220 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:18.383322 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:18.384121 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.385278 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.388498 (+ 3220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.389640 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.389803 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.389870 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:18.390580 (+ 710us) server_negotiation.cc:299] Negotiation successful
0809 12:56:18.390692 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 12:56:18.391299 24236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3061b4dec07d41f9a2e94fe500ed4cf1" candidate_uuid: "3ecc3d3f697b4402932082cd23fd42ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d7790219984fd3b1650b4e25f7429c" is_pre_election: true
I20260809 12:56:18.391712 24346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:18.381949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:36187 (local address 127.21.235.66:57071)
0809 12:56:18.382670 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:18.382692 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:18.382903 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:18.383356 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:18.383360 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:18.383384 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:18.383895 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:18.383906 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.385427 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.385433 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:18.388323 (+ 2890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:18.388338 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.390144 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.390154 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:18.390353 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.390395 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 12:56:18.391130 (+ 735us) client_negotiation.cc:240] Negotiation successful
0809 12:56:18.391267 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":71,"thread_start_us":341,"threads_started":1}
I20260809 12:56:18.391824 24236 raft_consensus.cc:2454] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ecc3d3f697b4402932082cd23fd42ee in term 0.
I20260809 12:56:18.392542 23997 leader_election.cc:302] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d7790219984fd3b1650b4e25f7429c, 3ecc3d3f697b4402932082cd23fd42ee; no voters:
I20260809 12:56:18.392846 23916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3061b4dec07d41f9a2e94fe500ed4cf1" candidate_uuid: "3ecc3d3f697b4402932082cd23fd42ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11da07ecf753480ba403b9318fa2eb27" is_pre_election: true
I20260809 12:56:18.392978 24320 raft_consensus.cc:2781] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:56:18.393083 24320 raft_consensus.cc:482] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:56:18.393097 23916 raft_consensus.cc:2454] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ecc3d3f697b4402932082cd23fd42ee in term 0.
I20260809 12:56:18.393142 24320 raft_consensus.cc:3037] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:18.395632 24320 raft_consensus.cc:504] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } }
I20260809 12:56:18.396286 24320 leader_election.cc:288] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [CANDIDATE]: Term 1 election: Requested vote from peers 11da07ecf753480ba403b9318fa2eb27 (127.21.235.65:36187), 06d7790219984fd3b1650b4e25f7429c (127.21.235.67:39805)
I20260809 12:56:18.396731 23916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3061b4dec07d41f9a2e94fe500ed4cf1" candidate_uuid: "3ecc3d3f697b4402932082cd23fd42ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11da07ecf753480ba403b9318fa2eb27"
I20260809 12:56:18.396894 24236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3061b4dec07d41f9a2e94fe500ed4cf1" candidate_uuid: "3ecc3d3f697b4402932082cd23fd42ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d7790219984fd3b1650b4e25f7429c"
I20260809 12:56:18.396966 23916 raft_consensus.cc:3037] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:18.397147 24236 raft_consensus.cc:3037] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:18.400223 23916 raft_consensus.cc:2454] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ecc3d3f697b4402932082cd23fd42ee in term 1.
I20260809 12:56:18.400655 23999 leader_election.cc:302] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11da07ecf753480ba403b9318fa2eb27, 3ecc3d3f697b4402932082cd23fd42ee; no voters:
I20260809 12:56:18.400938 24236 raft_consensus.cc:2454] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ecc3d3f697b4402932082cd23fd42ee in term 1.
I20260809 12:56:18.400977 24320 raft_consensus.cc:2781] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:18.401631 24320 raft_consensus.cc:686] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [term 1 LEADER]: Becoming Leader. State: Replica: 3ecc3d3f697b4402932082cd23fd42ee, State: Running, Role: LEADER
I20260809 12:56:18.402310 24320 consensus_queue.cc:237] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } }
I20260809 12:56:18.406701 23735 catalog_manager.cc:5495] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ecc3d3f697b4402932082cd23fd42ee (127.21.235.66). New cstate: current_term: 1 leader_uuid: "3ecc3d3f697b4402932082cd23fd42ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } health_report { overall_health: UNKNOWN } } }
Aug 09 12:56:18 dist-test-slave-gp6n krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280175, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:18.512250 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:18.481588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34686 (local address 127.21.235.126:45333)
0809 12:56:18.493501 (+ 11913us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:18.493508 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:18.494029 (+ 521us) server_negotiation.cc:407] Connection header received
0809 12:56:18.494287 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:18.494293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:18.494374 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:18.494518 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:18.496930 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.498184 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.500370 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.500903 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.501014 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.505739 (+ 4725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:18.505745 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:18.505748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:18.505778 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:18.509089 (+ 3311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:18.510054 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:18.510059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:18.510061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:18.510192 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:18.510693 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:18.510697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:18.510699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:18.511078 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:18.511191 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:18.511776 (+ 585us) server_negotiation.cc:299] Negotiation successful
0809 12:56:18.511956 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11714,"thread_start_us":11494,"threads_started":1}
I20260809 12:56:18.512714 24358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:18.481828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:45333 (local address 127.0.0.1:34686)
0809 12:56:18.493874 (+ 12046us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:18.493899 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:18.494131 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:18.494560 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:18.494569 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:18.496135 (+ 1566us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:18.496724 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:18.496740 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.498399 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.498448 (+ 49us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:18.500183 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:18.500199 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.501121 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.501129 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:18.501341 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.502151 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:18.502178 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:18.505459 (+ 3281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:18.509234 (+ 3775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:18.509248 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:18.509256 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:18.509912 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:18.510323 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:18.510328 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:18.510332 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:18.510585 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:18.511213 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:18.511223 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:18.511435 (+ 212us) client_negotiation.cc:769] Sending connection context
0809 12:56:18.511695 (+ 260us) client_negotiation.cc:240] Negotiation successful
0809 12:56:18.512178 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11787,"mutex_wait_us":110,"thread_start_us":11560,"threads_started":1}
I20260809 12:56:18.519459 23735 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:34686:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 12:56:18.520749 23735 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:56:18.529047 24051 tablet_service.cc:1467] Processing CreateTablet for tablet 2a4553621f6240a5b017c63b887b5d21 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ac9c99b49bfa4147adc4c30133addc69]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:56:18.529121 23887 tablet_service.cc:1467] Processing CreateTablet for tablet 2a4553621f6240a5b017c63b887b5d21 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ac9c99b49bfa4147adc4c30133addc69]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:56:18.529449 24208 tablet_service.cc:1467] Processing CreateTablet for tablet 2a4553621f6240a5b017c63b887b5d21 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ac9c99b49bfa4147adc4c30133addc69]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:56:18.529700 24051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a4553621f6240a5b017c63b887b5d21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:18.529753 23887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a4553621f6240a5b017c63b887b5d21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:18.529937 24208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a4553621f6240a5b017c63b887b5d21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:18.539736 24314 tablet_bootstrap.cc:492] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee: Bootstrap starting.
I20260809 12:56:18.541775 24314 tablet_bootstrap.cc:654] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:18.544605 24342 tablet_bootstrap.cc:492] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c: Bootstrap starting.
I20260809 12:56:18.547005 24342 tablet_bootstrap.cc:654] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:18.549247 24340 tablet_bootstrap.cc:492] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27: Bootstrap starting.
I20260809 12:56:18.551316 24340 tablet_bootstrap.cc:654] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:18.563280 24340 tablet_bootstrap.cc:492] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27: No bootstrap required, opened a new log
I20260809 12:56:18.563485 24340 ts_tablet_manager.cc:1397] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.006s
I20260809 12:56:18.564391 24340 raft_consensus.cc:348] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } }
I20260809 12:56:18.564723 24340 raft_consensus.cc:374] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:18.564816 24340 raft_consensus.cc:729] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11da07ecf753480ba403b9318fa2eb27, State: Initialized, Role: FOLLOWER
I20260809 12:56:18.565092 24340 consensus_queue.cc:260] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } }
I20260809 12:56:18.565689 24318 tablet_replica.cc:439] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } }
I20260809 12:56:18.566094 24318 tablet_replica.cc:442] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:56:18.566562 24340 ts_tablet_manager.cc:1428] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 12:56:18.569211 24314 tablet_bootstrap.cc:492] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee: No bootstrap required, opened a new log
I20260809 12:56:18.569512 24314 ts_tablet_manager.cc:1397] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee: Time spent bootstrapping tablet: real 0.030s user 0.007s sys 0.006s
I20260809 12:56:18.570441 24314 raft_consensus.cc:348] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } }
I20260809 12:56:18.570849 24314 raft_consensus.cc:374] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:18.570987 24314 raft_consensus.cc:729] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ecc3d3f697b4402932082cd23fd42ee, State: Initialized, Role: FOLLOWER
I20260809 12:56:18.571352 24342 tablet_bootstrap.cc:492] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c: No bootstrap required, opened a new log
I20260809 12:56:18.571583 24342 ts_tablet_manager.cc:1397] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c: Time spent bootstrapping tablet: real 0.027s user 0.008s sys 0.012s
I20260809 12:56:18.571413 24314 consensus_queue.cc:260] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } }
I20260809 12:56:18.572083 24320 tablet_replica.cc:439] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } }
I20260809 12:56:18.572504 24320 tablet_replica.cc:442] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:56:18.572453 24342 raft_consensus.cc:348] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } }
I20260809 12:56:18.572731 24342 raft_consensus.cc:374] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:18.572814 24342 raft_consensus.cc:729] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d7790219984fd3b1650b4e25f7429c, State: Initialized, Role: FOLLOWER
I20260809 12:56:18.573072 24342 consensus_queue.cc:260] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } }
I20260809 12:56:18.573609 24342 ts_tablet_manager.cc:1428] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 12:56:18.573490 24335 tablet_replica.cc:439] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } }
I20260809 12:56:18.573930 24335 tablet_replica.cc:442] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:56:18.575837 24314 ts_tablet_manager.cc:1428] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260809 12:56:18.649498 24335 raft_consensus.cc:482] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:56:18.649710 24335 raft_consensus.cc:504] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } }
I20260809 12:56:18.650507 24335 leader_election.cc:288] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11da07ecf753480ba403b9318fa2eb27 (127.21.235.65:36187), 3ecc3d3f697b4402932082cd23fd42ee (127.21.235.66:38077)
I20260809 12:56:18.651117 23916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4553621f6240a5b017c63b887b5d21" candidate_uuid: "06d7790219984fd3b1650b4e25f7429c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11da07ecf753480ba403b9318fa2eb27" is_pre_election: true
I20260809 12:56:18.651192 24072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4553621f6240a5b017c63b887b5d21" candidate_uuid: "06d7790219984fd3b1650b4e25f7429c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ecc3d3f697b4402932082cd23fd42ee" is_pre_election: true
I20260809 12:56:18.651424 23916 raft_consensus.cc:2454] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d7790219984fd3b1650b4e25f7429c in term 0.
I20260809 12:56:18.651480 24072 raft_consensus.cc:2454] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d7790219984fd3b1650b4e25f7429c in term 0.
I20260809 12:56:18.651940 24157 leader_election.cc:302] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d7790219984fd3b1650b4e25f7429c, 11da07ecf753480ba403b9318fa2eb27; no voters:
I20260809 12:56:18.652316 24335 raft_consensus.cc:2781] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:56:18.652452 24335 raft_consensus.cc:482] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:56:18.652542 24335 raft_consensus.cc:3037] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:18.656190 24335 raft_consensus.cc:504] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } }
I20260809 12:56:18.656901 24335 leader_election.cc:288] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [CANDIDATE]: Term 1 election: Requested vote from peers 11da07ecf753480ba403b9318fa2eb27 (127.21.235.65:36187), 3ecc3d3f697b4402932082cd23fd42ee (127.21.235.66:38077)
I20260809 12:56:18.657404 24072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4553621f6240a5b017c63b887b5d21" candidate_uuid: "06d7790219984fd3b1650b4e25f7429c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ecc3d3f697b4402932082cd23fd42ee"
I20260809 12:56:18.657442 23916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a4553621f6240a5b017c63b887b5d21" candidate_uuid: "06d7790219984fd3b1650b4e25f7429c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11da07ecf753480ba403b9318fa2eb27"
I20260809 12:56:18.657646 24072 raft_consensus.cc:3037] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:18.657682 23916 raft_consensus.cc:3037] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:18.660507 24072 raft_consensus.cc:2454] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d7790219984fd3b1650b4e25f7429c in term 1.
I20260809 12:56:18.660774 23916 raft_consensus.cc:2454] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d7790219984fd3b1650b4e25f7429c in term 1.
I20260809 12:56:18.661000 24156 leader_election.cc:302] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d7790219984fd3b1650b4e25f7429c, 3ecc3d3f697b4402932082cd23fd42ee; no voters:
I20260809 12:56:18.661371 24335 raft_consensus.cc:2781] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:18.661865 24335 raft_consensus.cc:686] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [term 1 LEADER]: Becoming Leader. State: Replica: 06d7790219984fd3b1650b4e25f7429c, State: Running, Role: LEADER
I20260809 12:56:18.662245 24335 consensus_queue.cc:237] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } }
I20260809 12:56:18.664072 24362 tablet_replica.cc:439] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c: TxnStatusTablet state changed. Reason: New leader 06d7790219984fd3b1650b4e25f7429c. Latest consensus state: current_term: 1 leader_uuid: "06d7790219984fd3b1650b4e25f7429c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } }
I20260809 12:56:18.664517 24362 tablet_replica.cc:442] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:56:18.665285 24364 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 12:56:18.665735 24364 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 12:56:18.665875 23735 catalog_manager.cc:5495] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c reported cstate change: term changed from 0 to 1, leader changed from <none> to 06d7790219984fd3b1650b4e25f7429c (127.21.235.67). New cstate: current_term: 1 leader_uuid: "06d7790219984fd3b1650b4e25f7429c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } health_report { overall_health: HEALTHY } } }
I20260809 12:56:18.713295 24307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:18.707065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35240 (local address 127.21.235.67:39805)
0809 12:56:18.707280 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:18.707288 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:18.707347 (+ 59us) server_negotiation.cc:407] Connection header received
0809 12:56:18.707652 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:18.707656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:18.707721 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:18.707801 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:56:18.708232 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.708961 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.711230 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.711822 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.711942 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.712586 (+ 644us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:56:18.712702 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:56:18.712791 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:18.712968 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 12:56:18.713050 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 12:56:18.713541 24358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:18.706983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:39805 (local address 127.0.0.1:35240)
0809 12:56:18.707266 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:18.707289 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:18.707452 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:18.707857 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:18.707862 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:18.707875 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:56:18.708096 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:18.708108 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.709097 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.709104 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:18.711012 (+ 1908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:18.711028 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.711990 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.711997 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:18.712256 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.712377 (+ 121us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:56:18.712812 (+ 435us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:56:18.712842 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 12:56:18.712940 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 12:56:18.713093 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 12:56:18.718883 24072 raft_consensus.cc:1261] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee [term 1 FOLLOWER]: Refusing update from remote peer 06d7790219984fd3b1650b4e25f7429c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:18.719010 23916 raft_consensus.cc:1261] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27 [term 1 FOLLOWER]: Refusing update from remote peer 06d7790219984fd3b1650b4e25f7429c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:18.719687 24362 consensus_queue.cc:1035] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:56:18.720084 24335 consensus_queue.cc:1035] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c [LEADER]: Connected to new peer: Peer: permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:56:18.729166 24318 tablet_replica.cc:439] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27: TxnStatusTablet state changed. Reason: New leader 06d7790219984fd3b1650b4e25f7429c. Latest consensus state: current_term: 1 leader_uuid: "06d7790219984fd3b1650b4e25f7429c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } }
I20260809 12:56:18.729673 24318 tablet_replica.cc:442] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:56:18.732321 24335 tablet_replica.cc:439] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "06d7790219984fd3b1650b4e25f7429c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } }
I20260809 12:56:18.732733 24335 tablet_replica.cc:442] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:56:18.732705 24369 mvcc.cc:205] Tried to move back new op lower bound from 7316603612025098240 to 7316603611802996736. Current Snapshot: MvccSnapshot[applied={T|T < 7316603612025098240}]
I20260809 12:56:18.736734 24366 mvcc.cc:205] Tried to move back new op lower bound from 7316603612025098240 to 7316603611802996736. Current Snapshot: MvccSnapshot[applied={T|T < 7316603612025098240 or (T in {7316603612025098240})}]
I20260809 12:56:18.737850 24318 tablet_replica.cc:439] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "06d7790219984fd3b1650b4e25f7429c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } }
I20260809 12:56:18.738195 24318 tablet_replica.cc:442] T 2a4553621f6240a5b017c63b887b5d21 P 11da07ecf753480ba403b9318fa2eb27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:56:18.743377 24320 tablet_replica.cc:439] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee: TxnStatusTablet state changed. Reason: New leader 06d7790219984fd3b1650b4e25f7429c. Latest consensus state: current_term: 1 leader_uuid: "06d7790219984fd3b1650b4e25f7429c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } }
I20260809 12:56:18.743762 24320 tablet_replica.cc:442] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:56:18.745446 24362 tablet_replica.cc:439] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "06d7790219984fd3b1650b4e25f7429c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } }
I20260809 12:56:18.745733 24362 tablet_replica.cc:442] T 2a4553621f6240a5b017c63b887b5d21 P 06d7790219984fd3b1650b4e25f7429c: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:56:18.747341 24368 mvcc.cc:205] Tried to move back new op lower bound from 7316603612025098240 to 7316603611802996736. Current Snapshot: MvccSnapshot[applied={T|T < 7316603612025098240}]
I20260809 12:56:18.760255 24320 tablet_replica.cc:439] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "06d7790219984fd3b1650b4e25f7429c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 } } peers { permanent_uuid: "3ecc3d3f697b4402932082cd23fd42ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38077 } } peers { permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 } } }
I20260809 12:56:18.761250 24320 tablet_replica.cc:442] T 2a4553621f6240a5b017c63b887b5d21 P 3ecc3d3f697b4402932082cd23fd42ee: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:56:18.770251 24376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:18.747084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39506 (local address 127.21.235.66:38077)
0809 12:56:18.763326 (+ 16242us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:18.763335 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:18.764022 (+ 687us) server_negotiation.cc:407] Connection header received
0809 12:56:18.764271 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:18.764280 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:18.764383 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:18.764558 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:56:18.765104 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.766000 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.767621 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.768345 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.768482 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.769033 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:56:18.769615 (+ 582us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:56:18.769773 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:18.769842 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 12:56:18.769920 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16031,"thread_start_us":16059,"threads_started":1}
I20260809 12:56:18.789191 24307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:18.782474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:34237 (local address 127.21.235.67:39805)
0809 12:56:18.782738 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:18.782744 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:18.783401 (+ 657us) server_negotiation.cc:407] Connection header received
0809 12:56:18.783599 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:18.783604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:18.783718 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:18.783809 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:56:18.784511 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.785426 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.787034 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.787726 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.787872 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.788323 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:56:18.788455 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:56:18.788553 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:18.788704 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 12:56:18.788784 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 12:56:18.789438 24380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:18.782516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:39805 (local address 127.21.235.66:34237)
0809 12:56:18.783193 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:18.783209 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:18.783461 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:18.783867 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:18.783873 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:18.783900 (+ 27us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:56:18.784349 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:18.784359 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.785574 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.785579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:18.786886 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:18.786898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.787932 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.787940 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:18.788070 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.788123 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:56:18.788576 (+ 453us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:56:18.788594 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 12:56:18.788681 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 12:56:18.788824 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":374,"threads_started":1}
I20260809 12:56:18.800541 24236 raft_consensus.cc:1261] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c [term 1 FOLLOWER]: Refusing update from remote peer 3ecc3d3f697b4402932082cd23fd42ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:18.800519 23915 raft_consensus.cc:1261] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27 [term 1 FOLLOWER]: Refusing update from remote peer 3ecc3d3f697b4402932082cd23fd42ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:18.801314 24320 consensus_queue.cc:1035] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "11da07ecf753480ba403b9318fa2eb27" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:56:18.801707 24348 consensus_queue.cc:1035] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d7790219984fd3b1650b4e25f7429c" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:56:18.842338 24376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:18.835611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:42083 (local address 127.21.235.66:38077)
0809 12:56:18.835783 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:18.835789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:18.836118 (+ 329us) server_negotiation.cc:407] Connection header received
0809 12:56:18.836588 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:18.836594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:18.836702 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:18.836791 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:56:18.837583 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.838701 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.840186 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.840793 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.840946 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.841471 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:56:18.841594 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:56:18.841668 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:18.841915 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 12:56:18.842002 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 12:56:18.843204 24384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:18.835167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:38077 (local address 127.21.235.67:42083)
0809 12:56:18.835891 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:18.835911 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:18.836231 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:18.836964 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:18.836968 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:18.836978 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:56:18.837368 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:18.837385 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.838841 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.838845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:18.840045 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:18.840057 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.840995 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.841005 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:18.841185 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.841260 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:56:18.841758 (+ 498us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:56:18.841773 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 12:56:18.842743 (+ 970us) client_negotiation.cc:240] Negotiation successful
0809 12:56:18.842887 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":635,"threads_started":1}
I20260809 12:56:18.901497 23737 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:56:18.901821 23737 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:56:18.905602 23737 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45c1b63f18934868b023be2d389b53e8: Sending DeleteTablet for 3 replicas of tablet 3061b4dec07d41f9a2e94fe500ed4cf1
I20260809 12:56:18.906556 23887 tablet_service.cc:1514] Processing DeleteTablet for tablet 3061b4dec07d41f9a2e94fe500ed4cf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:18 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:40474
I20260809 12:56:18.906675 24051 tablet_service.cc:1514] Processing DeleteTablet for tablet 3061b4dec07d41f9a2e94fe500ed4cf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:18 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:39496
I20260809 12:56:18.907768 24208 tablet_service.cc:1514] Processing DeleteTablet for tablet 3061b4dec07d41f9a2e94fe500ed4cf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:18 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:35216
I20260809 12:56:18.917580 24389 tablet_replica.cc:330] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27: stopping tablet replica
I20260809 12:56:18.917953 24389 raft_consensus.cc:2229] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:56:18.918278 24389 raft_consensus.cc:2258] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:18.919644 24389 ts_tablet_manager.cc:1905] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:18.928154 24389 ts_tablet_manager.cc:1918] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:56:18.928372 24389 log.cc:1196] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/3061b4dec07d41f9a2e94fe500ed4cf1
I20260809 12:56:18.928999 24389 ts_tablet_manager.cc:1939] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 11da07ecf753480ba403b9318fa2eb27: Deleting consensus metadata
I20260809 12:56:18.930434 23721 catalog_manager.cc:4841] TS 11da07ecf753480ba403b9318fa2eb27 (127.21.235.65:36187): tablet 3061b4dec07d41f9a2e94fe500ed4cf1 (table test-table [id=50a47e2e48b14407aa7e97611a0c8636]) successfully deleted
I20260809 12:56:18.935406 24392 tablet_replica.cc:330] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c: stopping tablet replica
I20260809 12:56:18.935731 24392 raft_consensus.cc:2229] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:56:18.936168 24392 raft_consensus.cc:2258] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:18.937520 24392 ts_tablet_manager.cc:1905] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:18.942895 24391 tablet_replica.cc:330] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee: stopping tablet replica
I20260809 12:56:18.943169 24391 raft_consensus.cc:2229] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:56:18.943804 24391 raft_consensus.cc:2258] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:18.945048 24391 ts_tablet_manager.cc:1905] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:18.953187 24391 ts_tablet_manager.cc:1918] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:56:18.956842 24392 ts_tablet_manager.cc:1918] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:56:18.957185 24392 log.cc:1196] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/3061b4dec07d41f9a2e94fe500ed4cf1
I20260809 12:56:18.957912 24392 ts_tablet_manager.cc:1939] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 06d7790219984fd3b1650b4e25f7429c: Deleting consensus metadata
I20260809 12:56:18.959311 23718 catalog_manager.cc:4841] TS 06d7790219984fd3b1650b4e25f7429c (127.21.235.67:39805): tablet 3061b4dec07d41f9a2e94fe500ed4cf1 (table test-table [id=50a47e2e48b14407aa7e97611a0c8636]) successfully deleted
I20260809 12:56:18.959802 24391 log.cc:1196] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/3061b4dec07d41f9a2e94fe500ed4cf1
I20260809 12:56:18.960548 24391 ts_tablet_manager.cc:1939] T 3061b4dec07d41f9a2e94fe500ed4cf1 P 3ecc3d3f697b4402932082cd23fd42ee: Deleting consensus metadata
I20260809 12:56:18.961756 23719 catalog_manager.cc:4841] TS 3ecc3d3f697b4402932082cd23fd42ee (127.21.235.66:38077): tablet 3061b4dec07d41f9a2e94fe500ed4cf1 (table test-table [id=50a47e2e48b14407aa7e97611a0c8636]) successfully deleted
Aug 09 12:56:18 dist-test-slave-gp6n krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280177, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 12:56:18.972755 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:18.935921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40478 (local address 127.21.235.65:36187)
0809 12:56:18.948997 (+ 13076us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:18.949006 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:18.949030 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:56:18.949100 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:18.949106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:18.949203 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:18.949361 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:18.952358 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.953783 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.956211 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.957014 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.962163 (+ 5149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.963993 (+ 1830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:18.964011 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:18.964032 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:18.964083 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:18.968711 (+ 4628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:18.969981 (+ 1270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:18.969991 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:18.969995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:18.970161 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:18.970760 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:18.970765 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:18.970768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:18.971318 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:18.971727 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:18.972066 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 12:56:18.972309 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12924,"thread_start_us":13589,"threads_started":1}
W20260809 12:56:18.974027 23967 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40478
I20260809 12:56:19.005509 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:18.978743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34692 (local address 127.21.235.126:45333)
0809 12:56:18.978941 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:18.978947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:18.987372 (+ 8425us) server_negotiation.cc:407] Connection header received
0809 12:56:18.987552 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:18.987559 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:18.987672 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:18.987803 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:18.989774 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.991121 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.993097 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:18.994064 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:18.994224 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:18.997866 (+ 3642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:18.997875 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:18.997881 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:18.997929 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:19.002195 (+ 4266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:19.003093 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:19.003100 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:19.003104 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:19.003287 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:19.003785 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:19.003791 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:19.003795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:19.004190 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:19.004493 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:19.004811 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 12:56:19.005113 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 12:56:19.006245 23737 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34692
I20260809 12:56:19.008944 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 23816
I20260809 12:56:19.041061 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:19.011247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40486 (local address 127.21.235.65:36187)
0809 12:56:19.011491 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:19.011497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:19.021454 (+ 9957us) server_negotiation.cc:407] Connection header received
0809 12:56:19.022102 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:19.022110 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:19.022207 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:19.022303 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:19.024314 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:19.025529 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:19.027674 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:19.028509 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:19.028648 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:19.032635 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:19.032642 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:19.032647 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:19.032691 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:19.036962 (+ 4271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:19.037975 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:19.037981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:19.037985 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:19.038151 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:19.038713 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:19.038718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:19.038722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:19.039176 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:19.039527 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:19.040259 (+ 732us) server_negotiation.cc:299] Negotiation successful
0809 12:56:19.040575 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
W20260809 12:56:19.043033 23967 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40486
W20260809 12:56:19.043869 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 23816: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:56:19.044106 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 23816
I20260809 12:56:19.075908 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 23977
Aug 09 12:56:19 dist-test-slave-gp6n krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280177, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.235.66@KRBTEST.COM
I20260809 12:56:19.098095 24376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:19.077114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39508 (local address 127.21.235.66:38077)
0809 12:56:19.077322 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:19.077332 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:19.077354 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:56:19.077448 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:19.077454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:19.077557 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:19.077692 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:19.079470 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:19.080847 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:19.082724 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:19.083483 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:19.083637 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:19.089026 (+ 5389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:19.089048 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:19.089071 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:19.089133 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:19.093932 (+ 4799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:19.095242 (+ 1310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:19.095257 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:19.095262 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:19.095486 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:19.095973 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:19.095979 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:19.095984 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:19.096589 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:19.096970 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:19.097323 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 12:56:19.097611 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 12:56:19.098963 24129 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39508
W20260809 12:56:19.099784 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 23977: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:56:19.099982 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 23977
I20260809 12:56:19.122128 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 24138
Aug 09 12:56:19 dist-test-slave-gp6n krb5kdc[23686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280177, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.235.67@KRBTEST.COM
I20260809 12:56:19.140583 24307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:19.123079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35250 (local address 127.21.235.67:39805)
0809 12:56:19.123301 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:19.123308 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:19.123327 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:56:19.123564 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:19.123568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:19.123634 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:19.123724 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:19.125439 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:19.126306 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:19.128292 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:19.128785 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:19.128907 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:19.134388 (+ 5481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:19.134547 (+ 159us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:19.134565 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:19.134610 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:19.137700 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:19.138546 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:19.138553 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:19.138555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:19.138692 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:19.139110 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:19.139114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:19.139115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:19.139480 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:19.139716 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:19.140063 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 12:56:19.140250 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260809 12:56:19.141383 24284 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35250
W20260809 12:56:19.142076 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 24138: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:56:19.142233 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 24138
I20260809 12:56:19.170136 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 23700
I20260809 12:56:19.171339 23782 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:19.288532 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 23700
2026-08-09T12:56:19Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5255 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:56:19 dist-test-slave-gp6n krb5kdc[24406](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:56:19 dist-test-slave-gp6n krb5kdc[24406](info): set up 2 sockets
Aug 09 12:56:19 dist-test-slave-gp6n krb5kdc[24406](info): commencing operation
krb5kdc: starting...
W20260809 12:56:20.472493 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.002s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:56:20 dist-test-slave-gp6n krb5kdc[24406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:56:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:56:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 12:56:20.682962 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:40617
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:41123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:40617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:21.030802 24421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:21.031136 24421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:21.031287 24421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:21.038575 24421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:56:21.038709 24421 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 12:56:21.038805 24421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:21.038921 24421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:21.039014 24421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:56:21.039105 24421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:56:21.049511 24421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:41123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:40617
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:40617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:21.050856 24421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:21.052214 24421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:21.064043 24426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:21.064148 24427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:21.064922 24430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:21.065200 24421 server_base.cc:1034] running on GCE node
I20260809 12:56:21.066013 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:21.067459 24421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:21.068677 24421 hybrid_clock.cc:648] HybridClock initialized: now 1786280181068621 us; error 37 us; skew 500 ppm
I20260809 12:56:21.069012 24421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:21 dist-test-slave-gp6n krb5kdc[24406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280181, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:21.072336 24421 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:56:21.073868 24421 webserver.cc:460] Webserver started at http://127.21.235.126:38533/ using document root <none> and password file <none>
I20260809 12:56:21.074653 24421 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:21.074787 24421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:21.075114 24421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:21.078032 24421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "db46cef39c2c4782aa9c9d2ece608b96"
format_stamp: "Formatted at 2026-08-09 12:56:21 on dist-test-slave-gp6n"
server_key: "6f373f46ce7dc7e0ade1d968f592917c"
server_key_iv: "37da0a351608d71d98846be879d43ce4"
server_key_version: "encryptionkey@0"
I20260809 12:56:21.078898 24421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "db46cef39c2c4782aa9c9d2ece608b96"
format_stamp: "Formatted at 2026-08-09 12:56:21 on dist-test-slave-gp6n"
server_key: "6f373f46ce7dc7e0ade1d968f592917c"
server_key_iv: "37da0a351608d71d98846be879d43ce4"
server_key_version: "encryptionkey@0"
I20260809 12:56:21.084127 24421 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:56:21.087601 24437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:21.088932 24421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:56:21.089125 24421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "db46cef39c2c4782aa9c9d2ece608b96"
format_stamp: "Formatted at 2026-08-09 12:56:21 on dist-test-slave-gp6n"
server_key: "6f373f46ce7dc7e0ade1d968f592917c"
server_key_iv: "37da0a351608d71d98846be879d43ce4"
server_key_version: "encryptionkey@0"
I20260809 12:56:21.089365 24421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:21.120000 24421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:21.124667 24421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:21.124959 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:21.147673 24421 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:40617
I20260809 12:56:21.147681 24505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:40617 every 8 connection(s)
I20260809 12:56:21.149041 24421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:56:21.152776 24507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:21.153331 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 24421
I20260809 12:56:21.153558 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:56:21.154233 22445 external_mini_cluster.cc:1428] Setting key 451d156ce457edca87cbf342dfb8bb56
I20260809 12:56:21.166726 24507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96: Bootstrap starting.
Aug 09 12:56:21 dist-test-slave-gp6n krb5kdc[24406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:21.170652 24507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:21.172200 24507 log.cc:824] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:21.176283 24507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96: No bootstrap required, opened a new log
I20260809 12:56:21.179153 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:21.156336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45768 (local address 127.21.235.126:40617)
0809 12:56:21.157185 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:21.157196 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:21.157241 (+ 45us) server_negotiation.cc:407] Connection header received
0809 12:56:21.159003 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:21.159025 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:21.159413 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:21.159747 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:21.161533 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:21.162930 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:21.164749 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:21.165408 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:21.165547 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:21.170252 (+ 4705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:21.170260 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:21.170274 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:21.170344 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:21.174897 (+ 4553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:21.175580 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:21.175588 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:21.175597 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:21.175791 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:21.176192 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:21.176197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:21.176201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:21.176836 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:21.177138 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:21.177604 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 12:56:21.178056 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":414,"threads_started":1}
I20260809 12:56:21.184103 24507 raft_consensus.cc:348] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db46cef39c2c4782aa9c9d2ece608b96" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40617 } }
I20260809 12:56:21.184469 24507 raft_consensus.cc:374] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:21.184581 24507 raft_consensus.cc:729] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db46cef39c2c4782aa9c9d2ece608b96, State: Initialized, Role: FOLLOWER
I20260809 12:56:21.185187 24507 consensus_queue.cc:260] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db46cef39c2c4782aa9c9d2ece608b96" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40617 } }
I20260809 12:56:21.185444 24507 raft_consensus.cc:388] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:56:21.185585 24507 raft_consensus.cc:482] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:56:21.185761 24507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:21.188359 24507 raft_consensus.cc:504] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db46cef39c2c4782aa9c9d2ece608b96" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40617 } }
I20260809 12:56:21.188825 24507 leader_election.cc:302] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db46cef39c2c4782aa9c9d2ece608b96; no voters:
I20260809 12:56:21.189411 24507 leader_election.cc:288] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:56:21.189505 24512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:21.190269 24512 raft_consensus.cc:686] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96 [term 1 LEADER]: Becoming Leader. State: Replica: db46cef39c2c4782aa9c9d2ece608b96, State: Running, Role: LEADER
I20260809 12:56:21.190793 24512 consensus_queue.cc:237] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db46cef39c2c4782aa9c9d2ece608b96" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40617 } }
I20260809 12:56:21.191505 24507 sys_catalog.cc:564] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:56:21.194121 24514 sys_catalog.cc:455] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db46cef39c2c4782aa9c9d2ece608b96. Latest consensus state: current_term: 1 leader_uuid: "db46cef39c2c4782aa9c9d2ece608b96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db46cef39c2c4782aa9c9d2ece608b96" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40617 } } }
I20260809 12:56:21.194631 24514 sys_catalog.cc:458] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96 [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:21.194995 24513 sys_catalog.cc:455] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db46cef39c2c4782aa9c9d2ece608b96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db46cef39c2c4782aa9c9d2ece608b96" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40617 } } }
I20260809 12:56:21.195621 24513 sys_catalog.cc:458] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96 [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:21.202080 24518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:56:21.210124 24518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:56:21.219820 24518 catalog_manager.cc:1347] Generated new cluster ID: 4d6e0b20cbb348b49148415783c4550e
I20260809 12:56:21.219969 24518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:56:21.239710 24518 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:56:21.241694 24518 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:56:21.249948 24518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96: Generated new TSK 0
I20260809 12:56:21.250993 24518 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
I20260809 12:56:21.310578 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40617
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:41123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:21.643332 24536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:21.643608 24536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:21.643716 24536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:21.650818 24536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:21.650942 24536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:21.651041 24536 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 12:56:21.651087 24536 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:56:21.651156 24536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:56:21.660878 24536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:41123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.235.126:40617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:21.662056 24536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:21.663493 24536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:21.676586 24542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:21.676687 24541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:21.676630 24546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:21.677541 24536 server_base.cc:1034] running on GCE node
I20260809 12:56:21.678207 24536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:21.679154 24536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:21.680389 24536 hybrid_clock.cc:648] HybridClock initialized: now 1786280181680348 us; error 37 us; skew 500 ppm
I20260809 12:56:21.680787 24536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:21 dist-test-slave-gp6n krb5kdc[24406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280181, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:21.684432 24536 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 12:56:21.686014 24536 webserver.cc:460] Webserver started at http://127.21.235.65:37807/ using document root <none> and password file <none>
I20260809 12:56:21.686743 24536 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:21.686895 24536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:21.687209 24536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:21.690150 24536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "0bf96d149d4149e4a0401ae92b0a0d54"
format_stamp: "Formatted at 2026-08-09 12:56:21 on dist-test-slave-gp6n"
server_key: "a45533ef01ab054b379cc39705182310"
server_key_iv: "aa8b4ec710449160df125b9cb631fca4"
server_key_version: "encryptionkey@0"
I20260809 12:56:21.690984 24536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "0bf96d149d4149e4a0401ae92b0a0d54"
format_stamp: "Formatted at 2026-08-09 12:56:21 on dist-test-slave-gp6n"
server_key: "a45533ef01ab054b379cc39705182310"
server_key_iv: "aa8b4ec710449160df125b9cb631fca4"
server_key_version: "encryptionkey@0"
I20260809 12:56:21.696496 24536 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260809 12:56:21.700044 24554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:21.701360 24536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:56:21.701519 24536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "0bf96d149d4149e4a0401ae92b0a0d54"
format_stamp: "Formatted at 2026-08-09 12:56:21 on dist-test-slave-gp6n"
server_key: "a45533ef01ab054b379cc39705182310"
server_key_iv: "aa8b4ec710449160df125b9cb631fca4"
server_key_version: "encryptionkey@0"
I20260809 12:56:21.701706 24536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:21.721519 24536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:21.725242 24536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:21.725548 24536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:21.728053 24536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:21.728201 24536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:21.728341 24536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:21.728446 24536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:56:21 dist-test-slave-gp6n krb5kdc[24406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280181, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:21.755486 24565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:21.729551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40617 (local address 127.21.235.65:37467)
0809 12:56:21.730538 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:21.730602 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:21.733443 (+ 2841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:21.734553 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:21.734561 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:21.735410 (+ 849us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:21.736367 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:21.736391 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:21.737591 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:21.737597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:21.739841 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:21.739856 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:21.741014 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:21.741025 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:21.741516 (+ 491us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:21.742336 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:21.742385 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:21.745701 (+ 3316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:21.749960 (+ 4259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:21.749969 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:21.749990 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:21.751000 (+ 1010us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:21.751473 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:21.751478 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:21.751482 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:21.751985 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:21.752754 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:21.752774 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:21.753332 (+ 558us) client_negotiation.cc:769] Sending connection context
0809 12:56:21.753661 (+ 329us) client_negotiation.cc:240] Negotiation successful
0809 12:56:21.754202 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":402,"threads_started":1}
I20260809 12:56:21.756669 24564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:21.728887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:37467 (local address 127.21.235.126:40617)
0809 12:56:21.729531 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:21.729538 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:21.730735 (+ 1197us) server_negotiation.cc:407] Connection header received
0809 12:56:21.733896 (+ 3161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:21.733904 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:21.734005 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:21.734220 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:21.736559 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:21.737456 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:21.740020 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:21.740653 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:21.740797 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:21.745981 (+ 5184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:21.745986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:21.745990 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:21.746024 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:21.749755 (+ 3731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:21.751173 (+ 1418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:21.751181 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:21.751187 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:21.751347 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:21.752113 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:21.752119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:21.752122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:21.752622 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:21.752784 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:21.756231 (+ 3447us) server_negotiation.cc:299] Negotiation successful
0809 12:56:21.756415 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":383,"threads_started":1}
I20260809 12:56:21.775507 24536 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:45555
I20260809 12:56:21.775534 24690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:45555 every 8 connection(s)
I20260809 12:56:21.776830 24536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:56:21.777590 24564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:21.763664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:35271 (local address 127.21.235.126:40617)
0809 12:56:21.763828 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:21.763832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:21.764572 (+ 740us) server_negotiation.cc:407] Connection header received
0809 12:56:21.764807 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:21.764814 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:21.764910 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:21.765013 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:21.766782 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:21.767784 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:21.769833 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:21.770244 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:21.770376 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:21.772185 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:21.772191 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:21.772196 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:21.772235 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:21.774874 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:21.775637 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:21.775642 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:21.775645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:21.775797 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:21.776169 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:21.776176 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:21.776179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:21.776451 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:21.776604 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:21.777025 (+ 421us) server_negotiation.cc:299] Negotiation successful
0809 12:56:21.777242 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 12:56:21.777848 24669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:21.763811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40617 (local address 127.21.235.65:35271)
0809 12:56:21.764450 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:21.764468 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:21.764660 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:21.765204 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:21.765209 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:21.766060 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:21.766563 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:21.766582 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:21.767939 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:21.767945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:21.769678 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:21.769690 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:21.770491 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:21.770500 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:21.770641 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:21.771272 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:21.771289 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:21.771993 (+ 704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:21.774997 (+ 3004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:21.775004 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:21.775007 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:21.775508 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:21.775912 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:21.775918 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:21.775922 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:21.776052 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:21.776571 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:21.776576 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:21.776764 (+ 188us) client_negotiation.cc:769] Sending connection context
0809 12:56:21.776911 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 12:56:21.777230 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":365,"threads_started":1}
I20260809 12:56:21.779472 24691 heartbeater.cc:347] Connected to a master server at 127.21.235.126:40617
I20260809 12:56:21.779736 24691 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:21.780081 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 24536
I20260809 12:56:21.780289 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:56:21.780623 24691 heartbeater.cc:511] Master 127.21.235.126:40617 requested a full tablet report, sending...
I20260809 12:56:21.781165 22445 external_mini_cluster.cc:1428] Setting key 8e7f19c52b812f611db6e9bd2f32093a
I20260809 12:56:21.783061 24456 ts_manager.cc:194] Registered new tserver with Master: 0bf96d149d4149e4a0401ae92b0a0d54 (127.21.235.65:45555)
I20260809 12:56:21.785198 24456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:37467
WARNING: no policy specified for kudu/127.21.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
I20260809 12:56:21.826525 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40617
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:41123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:22.173451 24695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:22.173804 24695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:22.173993 24695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:22.182895 24695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:22.183048 24695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:22.183202 24695 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 12:56:22.183297 24695 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:56:22.183411 24695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:56:22.193955 24695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:41123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.235.126:40617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:22.195365 24695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:22.196718 24695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:22.210510 24701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:22.210467 24702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:22.210536 24704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:22.211352 24695 server_base.cc:1034] running on GCE node
I20260809 12:56:22.211987 24695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:22.213013 24695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:22.214241 24695 hybrid_clock.cc:648] HybridClock initialized: now 1786280182214193 us; error 41 us; skew 500 ppm
I20260809 12:56:22.214598 24695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:22 dist-test-slave-gp6n krb5kdc[24406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280182, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:22.222370 24695 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 12:56:22.224169 24695 webserver.cc:460] Webserver started at http://127.21.235.66:37101/ using document root <none> and password file <none>
I20260809 12:56:22.225016 24695 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:22.225160 24695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:22.225497 24695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:22.228552 24695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "a4c009a4dee44df182a202600225ca3b"
format_stamp: "Formatted at 2026-08-09 12:56:22 on dist-test-slave-gp6n"
server_key: "246af30d49f223252393869c2f61d176"
server_key_iv: "6c656f4b6f5dab7c10c3255b82294b66"
server_key_version: "encryptionkey@0"
I20260809 12:56:22.229400 24695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "a4c009a4dee44df182a202600225ca3b"
format_stamp: "Formatted at 2026-08-09 12:56:22 on dist-test-slave-gp6n"
server_key: "246af30d49f223252393869c2f61d176"
server_key_iv: "6c656f4b6f5dab7c10c3255b82294b66"
server_key_version: "encryptionkey@0"
I20260809 12:56:22.235503 24695 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260809 12:56:22.239187 24712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:22.240545 24695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:56:22.240728 24695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "a4c009a4dee44df182a202600225ca3b"
format_stamp: "Formatted at 2026-08-09 12:56:22 on dist-test-slave-gp6n"
server_key: "246af30d49f223252393869c2f61d176"
server_key_iv: "6c656f4b6f5dab7c10c3255b82294b66"
server_key_version: "encryptionkey@0"
I20260809 12:56:22.240965 24695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:22.267349 24695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:22.271893 24695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:22.272192 24695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:22.275453 24695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:22.275584 24695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:22.275712 24695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:22.275802 24695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:56:22 dist-test-slave-gp6n krb5kdc[24406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280182, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:22.300882 24564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:22.276476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:49105 (local address 127.21.235.126:40617)
0809 12:56:22.276666 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:22.276673 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:22.278267 (+ 1594us) server_negotiation.cc:407] Connection header received
0809 12:56:22.280595 (+ 2328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:22.280602 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:22.280700 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:22.280828 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:22.282494 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.283512 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.286891 (+ 3379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.287543 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.287668 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:22.293020 (+ 5352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:22.293027 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:22.293032 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:22.293076 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:22.297058 (+ 3982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:22.297822 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:22.297831 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:22.297837 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:22.298030 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:22.298477 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:22.298485 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:22.298491 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:22.298918 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:22.299117 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:22.300159 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0809 12:56:22.300463 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:56:22.302538 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:22.277162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40617 (local address 127.21.235.66:49105)
0809 12:56:22.278065 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:22.278109 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:22.280240 (+ 2131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:22.281071 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:22.281079 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:22.281597 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:22.282234 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:22.282263 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.284259 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.284267 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:22.286654 (+ 2387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:22.286675 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.288112 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.288122 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:22.288365 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:22.289237 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:22.289288 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:22.292737 (+ 3449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:22.297213 (+ 4476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:22.297219 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:22.297264 (+ 45us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:22.297681 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:22.298147 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:22.298150 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:22.298152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:22.298338 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:22.299084 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:22.299095 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:22.299579 (+ 484us) client_negotiation.cc:769] Sending connection context
0809 12:56:22.299995 (+ 416us) client_negotiation.cc:240] Negotiation successful
0809 12:56:22.300659 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":459,"threads_started":1}
I20260809 12:56:22.326246 24564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:22.309412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:52249 (local address 127.21.235.126:40617)
0809 12:56:22.309673 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:22.309680 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:22.310255 (+ 575us) server_negotiation.cc:407] Connection header received
0809 12:56:22.310498 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:22.310505 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:22.310599 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:22.310713 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:22.312140 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.312844 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.315549 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.316113 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.316264 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:22.319083 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:22.319090 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:22.319095 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:22.319134 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:22.323058 (+ 3924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:22.323842 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:22.323849 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:22.323856 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:22.324053 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:22.324521 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:22.324527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:22.324530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:22.325027 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:22.325192 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:22.325615 (+ 423us) server_negotiation.cc:299] Negotiation successful
0809 12:56:22.325844 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 12:56:22.326324 24791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:22.309277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40617 (local address 127.21.235.66:52249)
0809 12:56:22.310019 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:22.310043 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:22.310346 (+ 303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:22.310760 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:22.310770 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:22.311516 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:22.311986 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:22.312003 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.312985 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.312993 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:22.315312 (+ 2319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:22.315335 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.316384 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.316396 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:22.316683 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:22.317436 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:22.317464 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:22.318826 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:22.323187 (+ 4361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:22.323191 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:22.323193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:22.323683 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:22.324200 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:22.324206 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:22.324209 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:22.324363 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:22.325161 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:22.325166 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:22.325351 (+ 185us) client_negotiation.cc:769] Sending connection context
0809 12:56:22.325498 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 12:56:22.325845 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":383,"threads_started":1}
I20260809 12:56:22.336071 24695 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:32835
I20260809 12:56:22.336091 24848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:32835 every 8 connection(s)
I20260809 12:56:22.337383 24695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:56:22.339534 24850 heartbeater.cc:347] Connected to a master server at 127.21.235.126:40617
I20260809 12:56:22.339771 24850 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:22.340420 24850 heartbeater.cc:511] Master 127.21.235.126:40617 requested a full tablet report, sending...
I20260809 12:56:22.341747 24456 ts_manager.cc:194] Registered new tserver with Master: a4c009a4dee44df182a202600225ca3b (127.21.235.66:32835)
I20260809 12:56:22.342684 24456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:49105
I20260809 12:56:22.346910 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 24695
I20260809 12:56:22.347095 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:56:22.347563 22445 external_mini_cluster.cc:1428] Setting key 0e40d92763d8090f09b9acb6054bfb5c
WARNING: no policy specified for kudu/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
I20260809 12:56:22.393813 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40617
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:41123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:22.722043 24856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:22.722366 24856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:22.722596 24856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:22.729686 24856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:22.729832 24856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:22.730000 24856 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 12:56:22.730091 24856 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:56:22.730207 24856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:56:22.740294 24856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:41123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.235.126:40617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:22.741590 24856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:22.743028 24856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:22.756716 24861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:22.756814 24862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:22.756754 24865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:22.757905 24856 server_base.cc:1034] running on GCE node
I20260809 12:56:22.758630 24856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:22.759711 24856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:22.760915 24856 hybrid_clock.cc:648] HybridClock initialized: now 1786280182760871 us; error 42 us; skew 500 ppm
I20260809 12:56:22.761232 24856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:22 dist-test-slave-gp6n krb5kdc[24406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280182, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:22.764509 24856 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 12:56:22.766043 24856 webserver.cc:460] Webserver started at http://127.21.235.67:41205/ using document root <none> and password file <none>
I20260809 12:56:22.766795 24856 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:22.766932 24856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:22.767261 24856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:22.770278 24856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "d4c97719025a45049ba3a2d0e40ab2ff"
format_stamp: "Formatted at 2026-08-09 12:56:22 on dist-test-slave-gp6n"
server_key: "c5d156500882bae23fc1d7305f3f2c93"
server_key_iv: "22fc7c9490892d35a0a2eb51be6bf5d4"
server_key_version: "encryptionkey@0"
I20260809 12:56:22.771335 24856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "d4c97719025a45049ba3a2d0e40ab2ff"
format_stamp: "Formatted at 2026-08-09 12:56:22 on dist-test-slave-gp6n"
server_key: "c5d156500882bae23fc1d7305f3f2c93"
server_key_iv: "22fc7c9490892d35a0a2eb51be6bf5d4"
server_key_version: "encryptionkey@0"
I20260809 12:56:22.777820 24856 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260809 12:56:22.781279 24873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:22.782681 24856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 12:56:22.782847 24856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "d4c97719025a45049ba3a2d0e40ab2ff"
format_stamp: "Formatted at 2026-08-09 12:56:22 on dist-test-slave-gp6n"
server_key: "c5d156500882bae23fc1d7305f3f2c93"
server_key_iv: "22fc7c9490892d35a0a2eb51be6bf5d4"
server_key_version: "encryptionkey@0"
I20260809 12:56:22.783059 24856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:22.788184 24691 heartbeater.cc:503] Master 127.21.235.126:40617 was elected leader, sending a full tablet report...
I20260809 12:56:22.802585 24856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:22.807353 24856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:22.807651 24856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:22.810356 24856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:22.810503 24856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 12:56:22.810631 24856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:22.810735 24856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:56:22 dist-test-slave-gp6n krb5kdc[24406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280182, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:22.835083 24564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:22.812348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:60867 (local address 127.21.235.126:40617)
0809 12:56:22.812539 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:22.812546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:22.812828 (+ 282us) server_negotiation.cc:407] Connection header received
0809 12:56:22.814998 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:22.815004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:22.815095 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:22.815217 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:22.817579 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.818724 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.821622 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.822063 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.822199 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:22.827134 (+ 4935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:22.827143 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:22.827151 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:22.827206 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:22.831292 (+ 4086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:22.832067 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:22.832076 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:22.832082 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:22.832302 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:22.832758 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:22.832765 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:22.832771 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:22.833267 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:22.833494 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:22.834161 (+ 667us) server_negotiation.cc:299] Negotiation successful
0809 12:56:22.834571 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 12:56:22.836278 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:22.811684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40617 (local address 127.21.235.67:60867)
0809 12:56:22.812647 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:22.812709 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:22.814699 (+ 1990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:22.815520 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:22.815535 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:22.816398 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:22.817365 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:22.817393 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.818908 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.818923 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:22.821454 (+ 2531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:22.821466 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.822339 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.822348 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:22.822726 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:22.823602 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:22.823643 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:22.826844 (+ 3201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:22.831473 (+ 4629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:22.831481 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:22.831494 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:22.831933 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:22.832432 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:22.832435 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:22.832437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:22.832640 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:22.833405 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:22.833411 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:22.833757 (+ 346us) client_negotiation.cc:769] Sending connection context
0809 12:56:22.833990 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 12:56:22.834395 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":392,"threads_started":1}
I20260809 12:56:22.860102 24564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:22.843852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:47657 (local address 127.21.235.126:40617)
0809 12:56:22.844073 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:22.844081 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:22.844905 (+ 824us) server_negotiation.cc:407] Connection header received
0809 12:56:22.845103 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:22.845107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:22.845168 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:22.845249 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:22.846926 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.847881 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.849976 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.850640 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.850785 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:22.853510 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:22.853518 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:22.853523 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:22.853570 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:22.857176 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:22.857873 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:22.857879 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:22.857883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:22.858021 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:22.858434 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:22.858442 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:22.858449 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:22.858870 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:22.859120 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:22.859586 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 12:56:22.859820 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:56:22.860731 24961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:22.844011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40617 (local address 127.21.235.67:47657)
0809 12:56:22.844743 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:22.844795 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:22.844979 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:22.845296 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:22.845303 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:22.846081 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:22.846664 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:22.846678 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.848035 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.848041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:22.849737 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:22.849751 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.851037 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.851047 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:22.851282 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:22.852258 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:22.852292 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:22.853261 (+ 969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:22.857313 (+ 4052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:22.857319 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:22.857323 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:22.857749 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:22.858137 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:22.858142 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:22.858146 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:22.858284 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:22.859076 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:22.859081 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:22.859316 (+ 235us) client_negotiation.cc:769] Sending connection context
0809 12:56:22.859486 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 12:56:22.860003 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":358,"threads_started":1}
I20260809 12:56:22.864811 24856 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:34405
I20260809 12:56:22.864827 25005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:34405 every 8 connection(s)
I20260809 12:56:22.866374 24856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:56:22.868729 25006 heartbeater.cc:347] Connected to a master server at 127.21.235.126:40617
I20260809 12:56:22.868958 25006 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:22.869601 25006 heartbeater.cc:511] Master 127.21.235.126:40617 requested a full tablet report, sending...
I20260809 12:56:22.870965 24456 ts_manager.cc:194] Registered new tserver with Master: d4c97719025a45049ba3a2d0e40ab2ff (127.21.235.67:34405)
I20260809 12:56:22.871801 24456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:60867
I20260809 12:56:22.874050 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 24856
I20260809 12:56:22.874228 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:56:22.874744 22445 external_mini_cluster.cc:1428] Setting key effb7c7a22a890c815ebfd1a751506b9
I20260809 12:56:22.877710 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:56:22 dist-test-slave-gp6n krb5kdc[24406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 12:56:22 dist-test-slave-gp6n krb5kdc[24406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:22.931227 24564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:22.916680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37076 (local address 127.21.235.126:40617)
0809 12:56:22.916943 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:22.916950 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:22.917218 (+ 268us) server_negotiation.cc:407] Connection header received
0809 12:56:22.917451 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:22.917456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:22.917523 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:22.917636 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:22.919048 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.919818 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.921081 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.921530 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.921670 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:22.925753 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:22.925759 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:22.925762 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:22.925797 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:22.928480 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:22.929150 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:22.929156 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:22.929160 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:22.929312 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:22.929704 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:22.929709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:22.929712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:22.930013 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:22.930281 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:22.930608 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 12:56:22.930851 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 12:56:22.935760 24456 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:56:22.942788 24456 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:56:22.981947 25025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:22.955516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:34405 (local address 127.0.0.1:40828)
0809 12:56:22.972494 (+ 16978us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:22.972561 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:22.972927 (+ 366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:22.974347 (+ 1420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:22.974358 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:22.974437 (+ 79us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:22.974819 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:22.974829 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.977208 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.977213 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:22.979136 (+ 1923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:22.979148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.980913 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.980919 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:22.981133 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:22.981181 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 12:56:22.981306 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 12:56:22.981486 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16602,"mutex_wait_us":1,"thread_start_us":18190,"threads_started":1}
I20260809 12:56:22.981972 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:22.955760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40828 (local address 127.21.235.67:34405)
0809 12:56:22.971394 (+ 15634us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:22.971406 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:22.972802 (+ 1396us) server_negotiation.cc:407] Connection header received
0809 12:56:22.973304 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:22.973317 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:22.973679 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:22.973908 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:22.975097 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.977105 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.979292 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.980400 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.980527 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:22.980760 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:22.981410 (+ 650us) server_negotiation.cc:299] Negotiation successful
0809 12:56:22.981607 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15422,"thread_start_us":15238,"threads_started":1}
I20260809 12:56:22.982918 25023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:22.956710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:32835 (local address 127.0.0.1:40266)
0809 12:56:22.974065 (+ 17355us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:22.974081 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:22.974225 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:22.974947 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:22.974952 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:22.974979 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:22.975401 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:22.975416 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.977252 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.977256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:22.979951 (+ 2695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:22.979967 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.982044 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.982049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:22.982236 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:22.982268 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 12:56:22.982356 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 12:56:22.982508 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17243,"mutex_wait_us":110,"thread_start_us":15216,"threads_started":1}
I20260809 12:56:22.982945 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:22.957337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40266 (local address 127.21.235.66:32835)
0809 12:56:22.971734 (+ 14397us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:22.971745 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:22.974195 (+ 2450us) server_negotiation.cc:407] Connection header received
0809 12:56:22.974387 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:22.974422 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:22.974658 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:22.974956 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:22.975630 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.977058 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.980120 (+ 3062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.981557 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.981695 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:22.981888 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:22.982480 (+ 592us) server_negotiation.cc:299] Negotiation successful
0809 12:56:22.982665 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14194,"thread_start_us":14058,"threads_started":1}
I20260809 12:56:22.983528 25022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:22.955981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45952 (local address 127.21.235.65:45555)
0809 12:56:22.974732 (+ 18751us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:22.974742 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:22.974784 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:56:22.974923 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:22.974930 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:22.975129 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:22.975394 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:22.975965 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.977432 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.980164 (+ 2732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.981676 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.981815 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:22.982026 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:22.983034 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0809 12:56:22.983212 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18590,"thread_start_us":18466,"threads_started":1}
I20260809 12:56:22.983928 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:22.955533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:45555 (local address 127.0.0.1:45952)
0809 12:56:22.973380 (+ 17847us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:22.973398 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:22.973579 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:22.975401 (+ 1822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:22.975408 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:22.975432 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:22.975797 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:22.975806 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.977589 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.977595 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:22.979960 (+ 2365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:22.979969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:22.982265 (+ 2296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:22.982270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:22.982506 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:22.982539 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 12:56:22.982632 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 12:56:22.982749 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17720,"mutex_wait_us":53,"thread_start_us":18358,"threads_started":1}
I20260809 12:56:22.986840 24769 tablet_service.cc:1467] Processing CreateTablet for tablet 63d294580913481f9fc6f9b4365160f9 (DEFAULT_TABLE table=test-table [id=ea75b1b8951640288de477bbb5f99445]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:22.986719 24928 tablet_service.cc:1467] Processing CreateTablet for tablet 63d294580913481f9fc6f9b4365160f9 (DEFAULT_TABLE table=test-table [id=ea75b1b8951640288de477bbb5f99445]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:22.987426 24608 tablet_service.cc:1467] Processing CreateTablet for tablet 63d294580913481f9fc6f9b4365160f9 (DEFAULT_TABLE table=test-table [id=ea75b1b8951640288de477bbb5f99445]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:22.988170 24769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63d294580913481f9fc6f9b4365160f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:22.988217 24928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63d294580913481f9fc6f9b4365160f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:22.988706 24608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63d294580913481f9fc6f9b4365160f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:22.999944 25027 tablet_bootstrap.cc:492] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b: Bootstrap starting.
I20260809 12:56:23.003217 25030 tablet_bootstrap.cc:492] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54: Bootstrap starting.
I20260809 12:56:23.003472 25027 tablet_bootstrap.cc:654] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:23.005038 25027 log.cc:824] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:23.005658 25031 tablet_bootstrap.cc:492] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff: Bootstrap starting.
I20260809 12:56:23.007285 25030 tablet_bootstrap.cc:654] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:23.008528 25030 log.cc:824] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:23.009059 25027 tablet_bootstrap.cc:492] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b: No bootstrap required, opened a new log
I20260809 12:56:23.009361 25031 tablet_bootstrap.cc:654] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:23.009409 25027 ts_tablet_manager.cc:1397] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260809 12:56:23.010623 25031 log.cc:824] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:23.011341 25030 tablet_bootstrap.cc:492] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54: No bootstrap required, opened a new log
I20260809 12:56:23.011617 25030 ts_tablet_manager.cc:1397] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 12:56:23.013617 25031 tablet_bootstrap.cc:492] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff: No bootstrap required, opened a new log
I20260809 12:56:23.013870 25031 ts_tablet_manager.cc:1397] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 12:56:23.019361 25030 raft_consensus.cc:348] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } }
I20260809 12:56:23.019755 25030 raft_consensus.cc:374] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:23.019881 25030 raft_consensus.cc:729] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf96d149d4149e4a0401ae92b0a0d54, State: Initialized, Role: FOLLOWER
I20260809 12:56:23.020234 25027 raft_consensus.cc:348] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } }
I20260809 12:56:23.020671 25027 raft_consensus.cc:374] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:23.020545 25030 consensus_queue.cc:260] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } }
I20260809 12:56:23.020780 25027 raft_consensus.cc:729] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4c009a4dee44df182a202600225ca3b, State: Initialized, Role: FOLLOWER
I20260809 12:56:23.021348 25027 consensus_queue.cc:260] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } }
I20260809 12:56:23.022661 24850 heartbeater.cc:503] Master 127.21.235.126:40617 was elected leader, sending a full tablet report...
I20260809 12:56:23.023211 25027 ts_tablet_manager.cc:1428] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260809 12:56:23.024300 25031 raft_consensus.cc:348] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } }
I20260809 12:56:23.024711 25031 raft_consensus.cc:374] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:23.024824 25031 raft_consensus.cc:729] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4c97719025a45049ba3a2d0e40ab2ff, State: Initialized, Role: FOLLOWER
I20260809 12:56:23.025399 25031 consensus_queue.cc:260] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } }
I20260809 12:56:23.026674 25006 heartbeater.cc:503] Master 127.21.235.126:40617 was elected leader, sending a full tablet report...
I20260809 12:56:23.027243 25031 ts_tablet_manager.cc:1428] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260809 12:56:23.028605 25030 ts_tablet_manager.cc:1428] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
W20260809 12:56:23.029860 24692 tablet.cc:2369] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:56:23.040690 25037 raft_consensus.cc:482] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:56:23.040962 25037 raft_consensus.cc:504] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } }
I20260809 12:56:23.042639 25037 leader_election.cc:288] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4c97719025a45049ba3a2d0e40ab2ff (127.21.235.67:34405), 0bf96d149d4149e4a0401ae92b0a0d54 (127.21.235.65:45555)
I20260809 12:56:23.050902 25022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.043220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:54813 (local address 127.21.235.65:45555)
0809 12:56:23.043420 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:23.043427 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:23.044317 (+ 890us) server_negotiation.cc:407] Connection header received
0809 12:56:23.044522 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:23.044528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:23.044638 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:23.044730 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:23.045645 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.046806 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.049037 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.049834 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.050052 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.050140 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:23.050516 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 12:56:23.050645 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:56:23.050951 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.043209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:48501 (local address 127.21.235.67:34405)
0809 12:56:23.043421 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:23.043427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:23.044307 (+ 880us) server_negotiation.cc:407] Connection header received
0809 12:56:23.044528 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:23.044533 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:23.044612 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:23.044720 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:23.045646 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.046839 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.049022 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.049819 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.050021 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.050111 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:23.050543 (+ 432us) server_negotiation.cc:299] Negotiation successful
0809 12:56:23.050673 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:56:23.051119 25040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.042989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:45555 (local address 127.21.235.66:54813)
0809 12:56:23.044203 (+ 1214us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:23.044221 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:23.044353 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:23.044746 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:23.044751 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:23.044792 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:23.045425 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.045445 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.046960 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.046965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.048870 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.048883 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.050130 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.050137 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.050316 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.050339 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 12:56:23.050486 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 12:56:23.050628 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1114,"mutex_wait_us":130,"spinlock_wait_cycles":4480,"thread_start_us":658,"threads_started":1}
I20260809 12:56:23.051412 25039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.042916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:34405 (local address 127.21.235.66:48501)
0809 12:56:23.044165 (+ 1249us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:23.044187 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:23.044398 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:23.044746 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:23.044751 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:23.044786 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:23.045507 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.045516 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.047015 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.047019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.048853 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.048870 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.050093 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.050100 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.050265 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.050337 (+ 72us) client_negotiation.cc:769] Sending connection context
0809 12:56:23.050503 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 12:56:23.050631 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1096,"mutex_wait_us":133,"spinlock_wait_cycles":42112,"thread_start_us":695,"threads_started":1}
I20260809 12:56:23.052461 24634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63d294580913481f9fc6f9b4365160f9" candidate_uuid: "a4c009a4dee44df182a202600225ca3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" is_pre_election: true
I20260809 12:56:23.052469 24957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63d294580913481f9fc6f9b4365160f9" candidate_uuid: "a4c009a4dee44df182a202600225ca3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" is_pre_election: true
I20260809 12:56:23.053078 24634 raft_consensus.cc:2454] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4c009a4dee44df182a202600225ca3b in term 0.
I20260809 12:56:23.053074 24957 raft_consensus.cc:2454] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4c009a4dee44df182a202600225ca3b in term 0.
I20260809 12:56:23.053946 24714 leader_election.cc:302] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0bf96d149d4149e4a0401ae92b0a0d54, a4c009a4dee44df182a202600225ca3b, d4c97719025a45049ba3a2d0e40ab2ff; no voters:
I20260809 12:56:23.054464 25037 raft_consensus.cc:2781] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:56:23.054641 25037 raft_consensus.cc:482] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:56:23.054775 25037 raft_consensus.cc:3037] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:23.057030 25037 raft_consensus.cc:504] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } }
I20260809 12:56:23.057642 25037 leader_election.cc:288] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [CANDIDATE]: Term 1 election: Requested vote from peers d4c97719025a45049ba3a2d0e40ab2ff (127.21.235.67:34405), 0bf96d149d4149e4a0401ae92b0a0d54 (127.21.235.65:45555)
I20260809 12:56:23.058257 24957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63d294580913481f9fc6f9b4365160f9" candidate_uuid: "a4c009a4dee44df182a202600225ca3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4c97719025a45049ba3a2d0e40ab2ff"
I20260809 12:56:23.058257 24634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63d294580913481f9fc6f9b4365160f9" candidate_uuid: "a4c009a4dee44df182a202600225ca3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf96d149d4149e4a0401ae92b0a0d54"
I20260809 12:56:23.058518 24634 raft_consensus.cc:3037] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:23.058506 24957 raft_consensus.cc:3037] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:23.061863 24634 raft_consensus.cc:2454] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4c009a4dee44df182a202600225ca3b in term 1.
I20260809 12:56:23.061863 24957 raft_consensus.cc:2454] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4c009a4dee44df182a202600225ca3b in term 1.
I20260809 12:56:23.062448 24715 leader_election.cc:302] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf96d149d4149e4a0401ae92b0a0d54, a4c009a4dee44df182a202600225ca3b; no voters:
I20260809 12:56:23.062815 25037 raft_consensus.cc:2781] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:23.063278 25037 raft_consensus.cc:686] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [term 1 LEADER]: Becoming Leader. State: Replica: a4c009a4dee44df182a202600225ca3b, State: Running, Role: LEADER
I20260809 12:56:23.063815 25037 consensus_queue.cc:237] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } }
I20260809 12:56:23.068645 24457 catalog_manager.cc:5495] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b reported cstate change: term changed from 0 to 1, leader changed from <none> to a4c009a4dee44df182a202600225ca3b (127.21.235.66). New cstate: current_term: 1 leader_uuid: "a4c009a4dee44df182a202600225ca3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } health_report { overall_health: HEALTHY } } }
W20260809 12:56:23.089430 24851 tablet.cc:2369] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:56:23.118048 25007 tablet.cc:2369] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 12:56:23 dist-test-slave-gp6n krb5kdc[24406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280181, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:23.127316 25051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.112851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40617 (local address 127.0.0.1:37080)
0809 12:56:23.113625 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:23.113642 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:23.113820 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:23.114236 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:23.114242 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:23.115207 (+ 965us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:23.115499 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.115510 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.116634 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.116643 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.117815 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.117831 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.118786 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.118794 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.119005 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.119629 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:23.119653 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:23.121915 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:23.124909 (+ 2994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:23.124920 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:23.124925 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:23.125346 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:23.125687 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:23.125690 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:23.125693 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:23.125845 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:23.126369 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:23.126375 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:23.126584 (+ 209us) client_negotiation.cc:769] Sending connection context
0809 12:56:23.126721 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 12:56:23.126924 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":387,"threads_started":1}
I20260809 12:56:23.127383 24564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.112855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37080 (local address 127.21.235.126:40617)
0809 12:56:23.113108 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:23.113117 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:23.113736 (+ 619us) server_negotiation.cc:407] Connection header received
0809 12:56:23.113993 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:23.113999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:23.114083 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:23.114198 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:23.115664 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.116494 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.117993 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.118571 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.118672 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.122099 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:23.122104 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:23.122108 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:23.122139 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:23.124787 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:23.125450 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:23.125454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:23.125456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:23.125590 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:23.125953 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:23.125957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:23.125959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:23.126255 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:23.126382 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:23.126830 (+ 448us) server_negotiation.cc:299] Negotiation successful
0809 12:56:23.127060 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":80}
I20260809 12:56:23.132511 24457 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:37080:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 12:56:23.133611 24457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:56:23.140442 24608 tablet_service.cc:1467] Processing CreateTablet for tablet 1ffae5d5b373445aa94c023ac8a13dec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9b0995c85a35493a9fe9abcddf901d8e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:56:23.140568 24769 tablet_service.cc:1467] Processing CreateTablet for tablet 1ffae5d5b373445aa94c023ac8a13dec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9b0995c85a35493a9fe9abcddf901d8e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:56:23.140592 24928 tablet_service.cc:1467] Processing CreateTablet for tablet 1ffae5d5b373445aa94c023ac8a13dec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9b0995c85a35493a9fe9abcddf901d8e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:56:23.141477 24608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ffae5d5b373445aa94c023ac8a13dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:23.141492 24928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ffae5d5b373445aa94c023ac8a13dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:23.141492 24769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ffae5d5b373445aa94c023ac8a13dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:23.147792 25030 tablet_bootstrap.cc:492] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54: Bootstrap starting.
I20260809 12:56:23.148855 25027 tablet_bootstrap.cc:492] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b: Bootstrap starting.
I20260809 12:56:23.149528 25030 tablet_bootstrap.cc:654] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:23.150389 25027 tablet_bootstrap.cc:654] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:23.150708 25031 tablet_bootstrap.cc:492] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff: Bootstrap starting.
I20260809 12:56:23.152918 25031 tablet_bootstrap.cc:654] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:23.160948 25031 tablet_bootstrap.cc:492] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff: No bootstrap required, opened a new log
I20260809 12:56:23.161160 25031 ts_tablet_manager.cc:1397] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260809 12:56:23.161250 25030 tablet_bootstrap.cc:492] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54: No bootstrap required, opened a new log
I20260809 12:56:23.161434 25030 ts_tablet_manager.cc:1397] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.011s
I20260809 12:56:23.162132 25031 raft_consensus.cc:348] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.162202 25030 raft_consensus.cc:348] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.162545 25031 raft_consensus.cc:374] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:23.162562 25030 raft_consensus.cc:374] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:23.162696 25030 raft_consensus.cc:729] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf96d149d4149e4a0401ae92b0a0d54, State: Initialized, Role: FOLLOWER
I20260809 12:56:23.162696 25031 raft_consensus.cc:729] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4c97719025a45049ba3a2d0e40ab2ff, State: Initialized, Role: FOLLOWER
I20260809 12:56:23.162979 25030 consensus_queue.cc:260] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.163574 25030 ts_tablet_manager.cc:1428] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 12:56:23.163719 25036 tablet_replica.cc:439] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } }
I20260809 12:56:23.164108 25036 tablet_replica.cc:442] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:56:23.164361 25031 consensus_queue.cc:260] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.164988 25038 tablet_replica.cc:439] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } }
I20260809 12:56:23.165686 25038 tablet_replica.cc:442] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:56:23.165036 25031 ts_tablet_manager.cc:1428] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 12:56:23.167659 25027 tablet_bootstrap.cc:492] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b: No bootstrap required, opened a new log
I20260809 12:56:23.167876 25027 ts_tablet_manager.cc:1397] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.017s
I20260809 12:56:23.168648 25027 raft_consensus.cc:348] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.168859 25027 raft_consensus.cc:374] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:23.168965 25027 raft_consensus.cc:729] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4c009a4dee44df182a202600225ca3b, State: Initialized, Role: FOLLOWER
I20260809 12:56:23.169210 25027 consensus_queue.cc:260] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.169659 25037 tablet_replica.cc:439] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } }
I20260809 12:56:23.169973 25037 tablet_replica.cc:442] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:56:23.170543 25027 ts_tablet_manager.cc:1428] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 12:56:23.223829 25038 raft_consensus.cc:482] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:56:23.224061 25038 raft_consensus.cc:504] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.225629 25038 leader_election.cc:288] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bf96d149d4149e4a0401ae92b0a0d54 (127.21.235.65:45555), a4c009a4dee44df182a202600225ca3b (127.21.235.66:32835)
I20260809 12:56:23.232779 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.226069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:56983 (local address 127.21.235.66:32835)
0809 12:56:23.226316 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:23.226323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:23.226530 (+ 207us) server_negotiation.cc:407] Connection header received
0809 12:56:23.226802 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:23.226809 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:23.226897 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:23.227011 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:23.227622 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.228574 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.230837 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.231878 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.232007 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.232068 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:23.232492 (+ 424us) server_negotiation.cc:299] Negotiation successful
0809 12:56:23.232583 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 12:56:23.233042 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.226069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:32835 (local address 127.21.235.67:56983)
0809 12:56:23.226376 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:23.226431 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:23.226611 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:23.227098 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:23.227106 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:23.227178 (+ 72us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:23.227457 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.227473 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.228721 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.228727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.230676 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.230689 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.232045 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.232053 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.232275 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.232359 (+ 84us) client_negotiation.cc:769] Sending connection context
0809 12:56:23.232477 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 12:56:23.232623 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":43,"spinlock_wait_cycles":36608}
I20260809 12:56:23.233827 25055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.226095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:45555 (local address 127.21.235.67:33165)
0809 12:56:23.227005 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:23.227023 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:23.227182 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:23.227589 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:23.227595 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:23.227620 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:23.228006 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.228018 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.229493 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.229499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.231371 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.231384 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.233000 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.233005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.233174 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.233209 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 12:56:23.233322 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 12:56:23.233473 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"mutex_wait_us":6,"spinlock_wait_cycles":22272,"thread_start_us":505,"threads_started":1}
I20260809 12:56:23.233857 25022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.226074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:33165 (local address 127.21.235.65:45555)
0809 12:56:23.226380 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:23.226385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:23.227151 (+ 766us) server_negotiation.cc:407] Connection header received
0809 12:56:23.227343 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:23.227348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:23.227438 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:23.227560 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:23.228181 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.229331 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.231556 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.232728 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.232921 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.233010 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:23.233424 (+ 414us) server_negotiation.cc:299] Negotiation successful
0809 12:56:23.233571 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180}
I20260809 12:56:23.234151 24796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ffae5d5b373445aa94c023ac8a13dec" candidate_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4c009a4dee44df182a202600225ca3b" is_pre_election: true
I20260809 12:56:23.234629 24634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ffae5d5b373445aa94c023ac8a13dec" candidate_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" is_pre_election: true
I20260809 12:56:23.234684 24796 raft_consensus.cc:2454] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4c97719025a45049ba3a2d0e40ab2ff in term 0.
I20260809 12:56:23.234903 24634 raft_consensus.cc:2454] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4c97719025a45049ba3a2d0e40ab2ff in term 0.
I20260809 12:56:23.235383 24877 leader_election.cc:302] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0bf96d149d4149e4a0401ae92b0a0d54, a4c009a4dee44df182a202600225ca3b, d4c97719025a45049ba3a2d0e40ab2ff; no voters:
I20260809 12:56:23.235841 25038 raft_consensus.cc:2781] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:56:23.235983 25038 raft_consensus.cc:482] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:56:23.236093 25038 raft_consensus.cc:3037] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:23.238444 25038 raft_consensus.cc:504] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.239033 25038 leader_election.cc:288] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [CANDIDATE]: Term 1 election: Requested vote from peers 0bf96d149d4149e4a0401ae92b0a0d54 (127.21.235.65:45555), a4c009a4dee44df182a202600225ca3b (127.21.235.66:32835)
I20260809 12:56:23.239491 24634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ffae5d5b373445aa94c023ac8a13dec" candidate_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf96d149d4149e4a0401ae92b0a0d54"
I20260809 12:56:23.239530 24796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ffae5d5b373445aa94c023ac8a13dec" candidate_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4c009a4dee44df182a202600225ca3b"
I20260809 12:56:23.239708 24634 raft_consensus.cc:3037] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:23.239713 24796 raft_consensus.cc:3037] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:23.241648 24634 raft_consensus.cc:2454] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4c97719025a45049ba3a2d0e40ab2ff in term 1.
I20260809 12:56:23.241814 24796 raft_consensus.cc:2454] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4c97719025a45049ba3a2d0e40ab2ff in term 1.
I20260809 12:56:23.242058 24876 leader_election.cc:302] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf96d149d4149e4a0401ae92b0a0d54, d4c97719025a45049ba3a2d0e40ab2ff; no voters:
I20260809 12:56:23.242372 25038 raft_consensus.cc:2781] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:23.242915 25038 raft_consensus.cc:686] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [term 1 LEADER]: Becoming Leader. State: Replica: d4c97719025a45049ba3a2d0e40ab2ff, State: Running, Role: LEADER
I20260809 12:56:23.243427 25038 consensus_queue.cc:237] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.246951 25056 tablet_replica.cc:439] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff: TxnStatusTablet state changed. Reason: New leader d4c97719025a45049ba3a2d0e40ab2ff. Latest consensus state: current_term: 1 leader_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } }
I20260809 12:56:23.247391 25056 tablet_replica.cc:442] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:56:23.248117 25058 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 12:56:23.248401 25058 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 12:56:23.248174 24455 catalog_manager.cc:5495] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff reported cstate change: term changed from 0 to 1, leader changed from <none> to d4c97719025a45049ba3a2d0e40ab2ff (127.21.235.67). New cstate: current_term: 1 leader_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } health_report { overall_health: HEALTHY } } }
I20260809 12:56:23.288461 25051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.281644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:34405 (local address 127.0.0.1:40832)
0809 12:56:23.281894 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:23.281921 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:23.282027 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:23.282591 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:23.282599 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:23.282610 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:56:23.282771 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.282778 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.284306 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.284310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.285337 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.285349 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.286517 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.286524 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.286748 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.286835 (+ 87us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:56:23.287965 (+ 1130us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:56:23.287984 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 12:56:23.288090 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 12:56:23.288216 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 12:56:23.288669 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.281831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40832 (local address 127.21.235.67:34405)
0809 12:56:23.282004 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:23.282010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:23.282031 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:56:23.282226 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:23.282232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:23.282334 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:23.282536 (+ 202us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:56:23.282914 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.284153 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.285503 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.286272 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.286472 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.287031 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:56:23.287834 (+ 803us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:56:23.288093 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:23.288161 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 12:56:23.288268 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:56:23.296561 24796 raft_consensus.cc:1261] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b [term 1 FOLLOWER]: Refusing update from remote peer d4c97719025a45049ba3a2d0e40ab2ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:23.296857 24634 raft_consensus.cc:1261] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Refusing update from remote peer d4c97719025a45049ba3a2d0e40ab2ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:23.297857 25038 consensus_queue.cc:1035] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:56:23.298265 25056 consensus_queue.cc:1035] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:56:23.311872 25036 tablet_replica.cc:439] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54: TxnStatusTablet state changed. Reason: New leader d4c97719025a45049ba3a2d0e40ab2ff. Latest consensus state: current_term: 1 leader_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } }
I20260809 12:56:23.312206 25036 tablet_replica.cc:442] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:56:23.314896 25037 tablet_replica.cc:439] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b: TxnStatusTablet state changed. Reason: New leader d4c97719025a45049ba3a2d0e40ab2ff. Latest consensus state: current_term: 1 leader_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } }
I20260809 12:56:23.315230 25037 tablet_replica.cc:442] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:56:23.317467 25062 mvcc.cc:205] Tried to move back new op lower bound from 7316603630772477952 to 7316603630568939520. Current Snapshot: MvccSnapshot[applied={T|T < 7316603630772477952}]
I20260809 12:56:23.318955 25061 mvcc.cc:205] Tried to move back new op lower bound from 7316603630772477952 to 7316603630568939520. Current Snapshot: MvccSnapshot[applied={T|T < 7316603630772477952}]
I20260809 12:56:23.325591 25037 tablet_replica.cc:439] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } }
I20260809 12:56:23.325898 25037 tablet_replica.cc:442] T 1ffae5d5b373445aa94c023ac8a13dec P a4c009a4dee44df182a202600225ca3b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:56:23.327349 25063 mvcc.cc:205] Tried to move back new op lower bound from 7316603630772477952 to 7316603630568939520. Current Snapshot: MvccSnapshot[applied={T|T < 7316603630772477952}]
I20260809 12:56:23.329289 25038 tablet_replica.cc:439] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } }
I20260809 12:56:23.329698 25038 tablet_replica.cc:442] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:56:23.333529 25038 tablet_replica.cc:439] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } }
I20260809 12:56:23.333863 25038 tablet_replica.cc:442] T 1ffae5d5b373445aa94c023ac8a13dec P d4c97719025a45049ba3a2d0e40ab2ff: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:56:23.346549 25036 tablet_replica.cc:439] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } } }
I20260809 12:56:23.346814 25036 tablet_replica.cc:442] T 1ffae5d5b373445aa94c023ac8a13dec P 0bf96d149d4149e4a0401ae92b0a0d54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:56:23.356297 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.350317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40272 (local address 127.21.235.66:32835)
0809 12:56:23.350532 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:23.350538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:23.350556 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:56:23.350650 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:23.350655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:23.350752 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:23.350912 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:56:23.351528 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.352578 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.353937 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.354365 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.354553 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.355010 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:56:23.355539 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:56:23.355714 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:23.355798 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 12:56:23.355891 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:56:23.380584 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.368008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:39557 (local address 127.21.235.67:34405)
0809 12:56:23.368278 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:23.368288 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:23.375090 (+ 6802us) server_negotiation.cc:407] Connection header received
0809 12:56:23.375286 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:23.375290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:23.375360 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:23.375445 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:56:23.376045 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.376765 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.378822 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.379298 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.379403 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.379968 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:56:23.380074 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:56:23.380146 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:23.380300 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 12:56:23.380368 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 12:56:23.380801 25072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.367908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:34405 (local address 127.21.235.66:39557)
0809 12:56:23.374962 (+ 7054us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:23.374988 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:23.375156 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:23.375487 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:23.375492 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:23.375506 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:56:23.375892 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.375904 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.376907 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.376912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.378680 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.378696 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.379485 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.379491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.379671 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.379742 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:56:23.380177 (+ 435us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:56:23.380188 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 12:56:23.380274 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 12:56:23.380419 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6866,"thread_start_us":6707,"threads_started":1}
I20260809 12:56:23.391738 24634 raft_consensus.cc:1261] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Refusing update from remote peer a4c009a4dee44df182a202600225ca3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:23.391783 24957 raft_consensus.cc:1261] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff [term 1 FOLLOWER]: Refusing update from remote peer a4c009a4dee44df182a202600225ca3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:23.392474 25041 consensus_queue.cc:1035] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:56:23.392853 25037 consensus_queue.cc:1035] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:56:23.433806 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.426883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:36931 (local address 127.21.235.66:32835)
0809 12:56:23.427093 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:23.427099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:23.427863 (+ 764us) server_negotiation.cc:407] Connection header received
0809 12:56:23.428086 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:23.428091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:23.428168 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:23.428261 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:56:23.428874 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.429992 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.431636 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.432408 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.432557 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.433060 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:56:23.433193 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:56:23.433277 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:23.433419 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 12:56:23.433496 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 12:56:23.433949 25078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.426954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:32835 (local address 127.21.235.67:36931)
0809 12:56:23.427756 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:23.427772 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:23.427940 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:23.428276 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:23.428281 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:23.428293 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:56:23.428679 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.428697 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.430150 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.430157 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.431486 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.431499 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.432623 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.432632 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.432812 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.432868 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:56:23.433278 (+ 410us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:56:23.433293 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 12:56:23.433397 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 12:56:23.433525 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":471,"threads_started":1}
I20260809 12:56:23.476297 24455 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:56:23.476645 24455 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:56:23.481065 24455 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96: Sending DeleteTablet for 3 replicas of tablet 63d294580913481f9fc6f9b4365160f9
I20260809 12:56:23.482448 24608 tablet_service.cc:1514] Processing DeleteTablet for tablet 63d294580913481f9fc6f9b4365160f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:23 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:45952
I20260809 12:56:23.482848 24928 tablet_service.cc:1514] Processing DeleteTablet for tablet 63d294580913481f9fc6f9b4365160f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:23 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:40828
I20260809 12:56:23.484184 24769 tablet_service.cc:1514] Processing DeleteTablet for tablet 63d294580913481f9fc6f9b4365160f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:23 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:40266
I20260809 12:56:23.486418 24455 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:56:23.487522 24455 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:56:23.492724 25083 tablet_replica.cc:330] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff: stopping tablet replica
I20260809 12:56:23.493328 25083 raft_consensus.cc:2229] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:56:23.493716 25083 raft_consensus.cc:2258] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:23.496618 25083 ts_tablet_manager.cc:1905] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:23.498745 24607 tablet_service.cc:1467] Processing CreateTablet for tablet ec962de967e44be28caaeb3ebd612430 (DEFAULT_TABLE table=test-table [id=e66ab6d1aca04d6c988053bfc21b2e16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:23.499353 24607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec962de967e44be28caaeb3ebd612430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:23.499857 24928 tablet_service.cc:1467] Processing CreateTablet for tablet ec962de967e44be28caaeb3ebd612430 (DEFAULT_TABLE table=test-table [id=e66ab6d1aca04d6c988053bfc21b2e16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:23.500475 24928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec962de967e44be28caaeb3ebd612430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:23.503217 24768 tablet_service.cc:1467] Processing CreateTablet for tablet ec962de967e44be28caaeb3ebd612430 (DEFAULT_TABLE table=test-table [id=e66ab6d1aca04d6c988053bfc21b2e16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:23.503925 24768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec962de967e44be28caaeb3ebd612430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:23.507818 25030 tablet_bootstrap.cc:492] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54: Bootstrap starting.
I20260809 12:56:23.509815 25030 tablet_bootstrap.cc:654] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:23.512220 25027 tablet_bootstrap.cc:492] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b: Bootstrap starting.
I20260809 12:56:23.514258 25027 tablet_bootstrap.cc:654] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:23.514880 25085 tablet_replica.cc:330] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b: stopping tablet replica
I20260809 12:56:23.515350 25085 raft_consensus.cc:2229] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:56:23.516027 25085 raft_consensus.cc:2258] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:23.520344 25085 ts_tablet_manager.cc:1905] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:23.521086 25083 ts_tablet_manager.cc:1918] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:56:23.524847 25083 log.cc:1196] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/63d294580913481f9fc6f9b4365160f9
I20260809 12:56:23.525514 25083 ts_tablet_manager.cc:1939] T 63d294580913481f9fc6f9b4365160f9 P d4c97719025a45049ba3a2d0e40ab2ff: Deleting consensus metadata
I20260809 12:56:23.528991 25030 tablet_bootstrap.cc:492] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54: No bootstrap required, opened a new log
I20260809 12:56:23.529184 25030 ts_tablet_manager.cc:1397] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.007s
I20260809 12:56:23.530018 25030 raft_consensus.cc:348] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.530287 25030 raft_consensus.cc:374] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:23.530383 25030 raft_consensus.cc:729] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf96d149d4149e4a0401ae92b0a0d54, State: Initialized, Role: FOLLOWER
I20260809 12:56:23.530665 25030 consensus_queue.cc:260] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.531306 25031 tablet_bootstrap.cc:492] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff: Bootstrap starting.
I20260809 12:56:23.531865 25030 ts_tablet_manager.cc:1428] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 12:56:23.533262 25031 tablet_bootstrap.cc:654] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:23.534605 24439 catalog_manager.cc:4841] TS d4c97719025a45049ba3a2d0e40ab2ff (127.21.235.67:34405): tablet 63d294580913481f9fc6f9b4365160f9 (table test-table [id=ea75b1b8951640288de477bbb5f99445]) successfully deleted
I20260809 12:56:23.536746 25027 tablet_bootstrap.cc:492] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b: No bootstrap required, opened a new log
I20260809 12:56:23.536947 25027 ts_tablet_manager.cc:1397] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.009s
I20260809 12:56:23.537788 25027 raft_consensus.cc:348] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.538216 25027 raft_consensus.cc:374] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:23.538324 25027 raft_consensus.cc:729] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4c009a4dee44df182a202600225ca3b, State: Initialized, Role: FOLLOWER
I20260809 12:56:23.538718 25027 consensus_queue.cc:260] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.542511 25027 ts_tablet_manager.cc:1428] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260809 12:56:23.547705 25031 tablet_bootstrap.cc:492] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff: No bootstrap required, opened a new log
I20260809 12:56:23.547895 25031 ts_tablet_manager.cc:1397] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.001s
I20260809 12:56:23.548519 25085 ts_tablet_manager.cc:1918] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:56:23.548791 25085 log.cc:1196] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/63d294580913481f9fc6f9b4365160f9
I20260809 12:56:23.548746 25031 raft_consensus.cc:348] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.549015 25031 raft_consensus.cc:374] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:23.549110 25031 raft_consensus.cc:729] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4c97719025a45049ba3a2d0e40ab2ff, State: Initialized, Role: FOLLOWER
I20260809 12:56:23.549636 25085 ts_tablet_manager.cc:1939] T 63d294580913481f9fc6f9b4365160f9 P a4c009a4dee44df182a202600225ca3b: Deleting consensus metadata
I20260809 12:56:23.549425 25031 consensus_queue.cc:260] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.551755 24441 catalog_manager.cc:4841] TS a4c009a4dee44df182a202600225ca3b (127.21.235.66:32835): tablet 63d294580913481f9fc6f9b4365160f9 (table test-table [id=ea75b1b8951640288de477bbb5f99445]) successfully deleted
I20260809 12:56:23.552397 25082 tablet_replica.cc:330] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54: stopping tablet replica
I20260809 12:56:23.552904 25082 raft_consensus.cc:2229] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:56:23.553217 25031 ts_tablet_manager.cc:1428] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 12:56:23.553339 25082 raft_consensus.cc:2258] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:23.555688 25082 ts_tablet_manager.cc:1905] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:23.561929 25082 ts_tablet_manager.cc:1918] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:56:23.562124 25082 log.cc:1196] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/63d294580913481f9fc6f9b4365160f9
I20260809 12:56:23.562713 25082 ts_tablet_manager.cc:1939] T 63d294580913481f9fc6f9b4365160f9 P 0bf96d149d4149e4a0401ae92b0a0d54: Deleting consensus metadata
I20260809 12:56:23.564293 24440 catalog_manager.cc:4841] TS 0bf96d149d4149e4a0401ae92b0a0d54 (127.21.235.65:45555): tablet 63d294580913481f9fc6f9b4365160f9 (table test-table [id=ea75b1b8951640288de477bbb5f99445]) successfully deleted
I20260809 12:56:23.565837 25036 raft_consensus.cc:482] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:56:23.566030 25036 raft_consensus.cc:504] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.567734 25036 leader_election.cc:288] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4c009a4dee44df182a202600225ca3b (127.21.235.66:32835), d4c97719025a45049ba3a2d0e40ab2ff (127.21.235.67:34405)
I20260809 12:56:23.586532 25041 raft_consensus.cc:482] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:56:23.586802 25041 raft_consensus.cc:504] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.587597 25041 leader_election.cc:288] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bf96d149d4149e4a0401ae92b0a0d54 (127.21.235.65:45555), d4c97719025a45049ba3a2d0e40ab2ff (127.21.235.67:34405)
I20260809 12:56:23.588608 24957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec962de967e44be28caaeb3ebd612430" candidate_uuid: "a4c009a4dee44df182a202600225ca3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" is_pre_election: true
I20260809 12:56:23.588871 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.569065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:35829 (local address 127.21.235.66:32835)
0809 12:56:23.569340 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:23.569349 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:23.579106 (+ 9757us) server_negotiation.cc:407] Connection header received
0809 12:56:23.579483 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:23.579493 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:23.579607 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:23.579692 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:23.580849 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.582182 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.585377 (+ 3195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.586971 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.587137 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.587219 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:23.588340 (+ 1121us) server_negotiation.cc:299] Negotiation successful
0809 12:56:23.588498 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 12:56:23.588904 24957 raft_consensus.cc:2454] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4c009a4dee44df182a202600225ca3b in term 0.
I20260809 12:56:23.589474 24714 leader_election.cc:302] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4c009a4dee44df182a202600225ca3b, d4c97719025a45049ba3a2d0e40ab2ff; no voters:
I20260809 12:56:23.589938 25041 raft_consensus.cc:2781] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:56:23.590058 25041 raft_consensus.cc:482] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:56:23.590183 25041 raft_consensus.cc:3037] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:23.591159 24632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec962de967e44be28caaeb3ebd612430" candidate_uuid: "a4c009a4dee44df182a202600225ca3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" is_pre_election: true
I20260809 12:56:23.591470 24632 raft_consensus.cc:2454] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4c009a4dee44df182a202600225ca3b in term 0.
I20260809 12:56:23.593964 25041 raft_consensus.cc:504] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.595188 24632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec962de967e44be28caaeb3ebd612430" candidate_uuid: "a4c009a4dee44df182a202600225ca3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf96d149d4149e4a0401ae92b0a0d54"
I20260809 12:56:23.595415 24632 raft_consensus.cc:3037] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:23.596761 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.568617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:53929 (local address 127.21.235.67:34405)
0809 12:56:23.568861 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:23.568871 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:23.581611 (+ 12740us) server_negotiation.cc:407] Connection header received
0809 12:56:23.582027 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:23.582034 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:23.582117 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:23.582190 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:23.583306 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.584652 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.593497 (+ 8845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.595222 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.595381 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.595473 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:23.596221 (+ 748us) server_negotiation.cc:299] Negotiation successful
0809 12:56:23.596403 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 12:56:23.597710 25091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.568131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:34405 (local address 127.21.235.65:53929)
0809 12:56:23.581499 (+ 13368us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:23.581520 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:23.581852 (+ 332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:23.582431 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:23.582435 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:23.582460 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:23.583045 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.583060 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.584932 (+ 1872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.584937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.593272 (+ 8335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.593299 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.595791 (+ 2492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.595799 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.595999 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.596038 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 12:56:23.597005 (+ 967us) client_negotiation.cc:240] Negotiation successful
0809 12:56:23.597179 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13236,"mutex_wait_us":32,"thread_start_us":13482,"threads_started":1}
I20260809 12:56:23.598453 24957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec962de967e44be28caaeb3ebd612430" candidate_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" is_pre_election: true
I20260809 12:56:23.598721 24957 raft_consensus.cc:2454] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bf96d149d4149e4a0401ae92b0a0d54 in term 0.
I20260809 12:56:23.598914 24632 raft_consensus.cc:2454] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4c009a4dee44df182a202600225ca3b in term 1.
I20260809 12:56:23.599326 24715 leader_election.cc:302] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf96d149d4149e4a0401ae92b0a0d54, a4c009a4dee44df182a202600225ca3b; no voters:
I20260809 12:56:23.599323 24556 leader_election.cc:302] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf96d149d4149e4a0401ae92b0a0d54, d4c97719025a45049ba3a2d0e40ab2ff; no voters:
I20260809 12:56:23.599828 25037 raft_consensus.cc:2781] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:23.600008 25037 raft_consensus.cc:686] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [term 1 LEADER]: Becoming Leader. State: Replica: a4c009a4dee44df182a202600225ca3b, State: Running, Role: LEADER
I20260809 12:56:23.600425 25041 leader_election.cc:288] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [CANDIDATE]: Term 1 election: Requested vote from peers 0bf96d149d4149e4a0401ae92b0a0d54 (127.21.235.65:45555), d4c97719025a45049ba3a2d0e40ab2ff (127.21.235.67:34405)
I20260809 12:56:23.600363 25037 consensus_queue.cc:237] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:23.601532 25036 raft_consensus.cc:2741] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260809 12:56:23.601707 25090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:23.567942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:32835 (local address 127.21.235.65:35829)
0809 12:56:23.578948 (+ 11006us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:23.578978 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:23.579258 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:23.579981 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:23.579989 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:23.580025 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:23.580591 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.580609 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.582394 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.582459 (+ 65us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.585185 (+ 2726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:23.585199 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:23.587912 (+ 2713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:23.587919 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:23.588126 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:23.588167 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 12:56:23.601064 (+ 12897us) client_negotiation.cc:240] Negotiation successful
0809 12:56:23.601223 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10786,"thread_start_us":10492,"threads_started":1}
I20260809 12:56:23.602459 24957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec962de967e44be28caaeb3ebd612430" candidate_uuid: "a4c009a4dee44df182a202600225ca3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4c97719025a45049ba3a2d0e40ab2ff"
I20260809 12:56:23.602607 24795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec962de967e44be28caaeb3ebd612430" candidate_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4c009a4dee44df182a202600225ca3b" is_pre_election: true
I20260809 12:56:23.602700 24957 raft_consensus.cc:3037] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:23.606061 24957 raft_consensus.cc:2454] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4c009a4dee44df182a202600225ca3b in term 1.
I20260809 12:56:23.614773 24455 catalog_manager.cc:5495] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b reported cstate change: term changed from 0 to 1, leader changed from <none> to a4c009a4dee44df182a202600225ca3b (127.21.235.66). New cstate: current_term: 1 leader_uuid: "a4c009a4dee44df182a202600225ca3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } health_report { overall_health: UNKNOWN } } }
I20260809 12:56:23.659034 24632 raft_consensus.cc:1261] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Refusing update from remote peer a4c009a4dee44df182a202600225ca3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:23.659803 25037 consensus_queue.cc:1035] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:56:23.662266 24957 raft_consensus.cc:1261] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff [term 1 FOLLOWER]: Refusing update from remote peer a4c009a4dee44df182a202600225ca3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:23.662988 25037 consensus_queue.cc:1035] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 12:56:23.697679 25063 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 12:56:25.305179 24455 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:56:25.305503 24455 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:56:25.309922 24455 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96: Sending DeleteTablet for 3 replicas of tablet ec962de967e44be28caaeb3ebd612430
I20260809 12:56:25.310739 24769 tablet_service.cc:1514] Processing DeleteTablet for tablet ec962de967e44be28caaeb3ebd612430 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:25 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:40266
I20260809 12:56:25.310698 24608 tablet_service.cc:1514] Processing DeleteTablet for tablet ec962de967e44be28caaeb3ebd612430 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:25 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:45952
I20260809 12:56:25.310822 24928 tablet_service.cc:1514] Processing DeleteTablet for tablet ec962de967e44be28caaeb3ebd612430 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:25 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:40828
I20260809 12:56:25.321812 25126 tablet_replica.cc:330] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff: stopping tablet replica
I20260809 12:56:25.322153 25126 raft_consensus.cc:2229] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:56:25.322530 25126 raft_consensus.cc:2258] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:25.324149 25126 ts_tablet_manager.cc:1905] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:25.326912 25123 tablet_replica.cc:330] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54: stopping tablet replica
I20260809 12:56:25.327240 25123 raft_consensus.cc:2229] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:56:25.327584 25123 raft_consensus.cc:2258] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:25.328328 25125 tablet_replica.cc:330] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b: stopping tablet replica
I20260809 12:56:25.328573 25125 raft_consensus.cc:2229] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:56:25.329012 25125 raft_consensus.cc:2258] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:25.330286 25123 ts_tablet_manager.cc:1905] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:25.331079 25125 ts_tablet_manager.cc:1905] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:25.334720 25126 ts_tablet_manager.cc:1918] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 12:56:25.334920 25126 log.cc:1196] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/ec962de967e44be28caaeb3ebd612430
I20260809 12:56:25.335472 25126 ts_tablet_manager.cc:1939] T ec962de967e44be28caaeb3ebd612430 P d4c97719025a45049ba3a2d0e40ab2ff: Deleting consensus metadata
I20260809 12:56:25.338620 24439 catalog_manager.cc:4841] TS d4c97719025a45049ba3a2d0e40ab2ff (127.21.235.67:34405): tablet ec962de967e44be28caaeb3ebd612430 (table test-table [id=e66ab6d1aca04d6c988053bfc21b2e16]) successfully deleted
I20260809 12:56:25.340566 25125 ts_tablet_manager.cc:1918] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 12:56:25.340729 25125 log.cc:1196] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/ec962de967e44be28caaeb3ebd612430
I20260809 12:56:25.341257 25125 ts_tablet_manager.cc:1939] T ec962de967e44be28caaeb3ebd612430 P a4c009a4dee44df182a202600225ca3b: Deleting consensus metadata
I20260809 12:56:25.342458 24441 catalog_manager.cc:4841] TS a4c009a4dee44df182a202600225ca3b (127.21.235.66:32835): tablet ec962de967e44be28caaeb3ebd612430 (table test-table [id=e66ab6d1aca04d6c988053bfc21b2e16]) successfully deleted
I20260809 12:56:25.344182 25123 ts_tablet_manager.cc:1918] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 12:56:25.344794 25123 log.cc:1196] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/ec962de967e44be28caaeb3ebd612430
I20260809 12:56:25.345317 25123 ts_tablet_manager.cc:1939] T ec962de967e44be28caaeb3ebd612430 P 0bf96d149d4149e4a0401ae92b0a0d54: Deleting consensus metadata
I20260809 12:56:25.346524 24440 catalog_manager.cc:4841] TS 0bf96d149d4149e4a0401ae92b0a0d54 (127.21.235.65:45555): tablet ec962de967e44be28caaeb3ebd612430 (table test-table [id=e66ab6d1aca04d6c988053bfc21b2e16]) successfully deleted
Aug 09 12:56:25 dist-test-slave-gp6n krb5kdc[24406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280185, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 12:56:25 dist-test-slave-gp6n krb5kdc[24406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280185, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:25.384133 25137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:25.369147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37086 (local address 127.21.235.126:40617)
0809 12:56:25.369919 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:25.369926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:25.370041 (+ 115us) server_negotiation.cc:407] Connection header received
0809 12:56:25.370270 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:25.370274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:25.370339 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:25.370475 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:25.371902 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:25.372683 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:25.373967 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:25.374376 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:25.374562 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:25.378522 (+ 3960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:25.378527 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:25.378531 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:25.378564 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:25.381319 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:25.381963 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:25.381970 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:25.381974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:25.382133 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:25.382611 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:25.382616 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:25.382619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:25.382905 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:25.383183 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:25.383503 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 12:56:25.383760 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":418,"threads_started":1}
I20260809 12:56:25.388021 24455 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:56:25.388644 24455 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:56:25.395186 24769 tablet_service.cc:1467] Processing CreateTablet for tablet dccbb5300d784157a143a2d83986be2c (DEFAULT_TABLE table=test-table [id=08e4b175f0184e2fab64512ad371defd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:25.395160 24608 tablet_service.cc:1467] Processing CreateTablet for tablet dccbb5300d784157a143a2d83986be2c (DEFAULT_TABLE table=test-table [id=08e4b175f0184e2fab64512ad371defd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:25.395411 24928 tablet_service.cc:1467] Processing CreateTablet for tablet dccbb5300d784157a143a2d83986be2c (DEFAULT_TABLE table=test-table [id=08e4b175f0184e2fab64512ad371defd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:25.395632 24769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dccbb5300d784157a143a2d83986be2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:25.395640 24608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dccbb5300d784157a143a2d83986be2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:25.395884 24928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dccbb5300d784157a143a2d83986be2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:25.404672 25138 tablet_bootstrap.cc:492] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff: Bootstrap starting.
I20260809 12:56:25.406769 25138 tablet_bootstrap.cc:654] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:25.408473 25139 tablet_bootstrap.cc:492] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54: Bootstrap starting.
I20260809 12:56:25.409850 25138 tablet_bootstrap.cc:492] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff: No bootstrap required, opened a new log
I20260809 12:56:25.410043 25138 ts_tablet_manager.cc:1397] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 12:56:25.410633 25140 tablet_bootstrap.cc:492] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b: Bootstrap starting.
I20260809 12:56:25.410890 25138 raft_consensus.cc:348] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:25.411113 25138 raft_consensus.cc:374] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:25.411242 25138 raft_consensus.cc:729] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4c97719025a45049ba3a2d0e40ab2ff, State: Initialized, Role: FOLLOWER
I20260809 12:56:25.411520 25138 consensus_queue.cc:260] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:25.412038 25138 ts_tablet_manager.cc:1428] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 12:56:25.412621 25140 tablet_bootstrap.cc:654] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:25.413326 25139 tablet_bootstrap.cc:654] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:25.416164 25140 tablet_bootstrap.cc:492] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b: No bootstrap required, opened a new log
I20260809 12:56:25.416438 25140 ts_tablet_manager.cc:1397] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260809 12:56:25.417610 25139 tablet_bootstrap.cc:492] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54: No bootstrap required, opened a new log
I20260809 12:56:25.417482 25140 raft_consensus.cc:348] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:25.417773 25140 raft_consensus.cc:374] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:25.417837 25139 ts_tablet_manager.cc:1397] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.004s
I20260809 12:56:25.417928 25140 raft_consensus.cc:729] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4c009a4dee44df182a202600225ca3b, State: Initialized, Role: FOLLOWER
I20260809 12:56:25.418265 25140 consensus_queue.cc:260] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:25.418948 25139 raft_consensus.cc:348] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:25.419178 25140 ts_tablet_manager.cc:1428] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 12:56:25.419324 25139 raft_consensus.cc:374] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:25.419472 25139 raft_consensus.cc:729] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf96d149d4149e4a0401ae92b0a0d54, State: Initialized, Role: FOLLOWER
I20260809 12:56:25.419741 25139 consensus_queue.cc:260] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:25.420842 25139 ts_tablet_manager.cc:1428] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260809 12:56:25.617164 25117 raft_consensus.cc:482] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:56:25.617420 25117 raft_consensus.cc:504] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:25.618278 25117 leader_election.cc:288] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bf96d149d4149e4a0401ae92b0a0d54 (127.21.235.65:45555), d4c97719025a45049ba3a2d0e40ab2ff (127.21.235.67:34405)
I20260809 12:56:25.618878 24632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dccbb5300d784157a143a2d83986be2c" candidate_uuid: "a4c009a4dee44df182a202600225ca3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" is_pre_election: true
I20260809 12:56:25.618928 24956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dccbb5300d784157a143a2d83986be2c" candidate_uuid: "a4c009a4dee44df182a202600225ca3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" is_pre_election: true
I20260809 12:56:25.619140 24632 raft_consensus.cc:2454] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4c009a4dee44df182a202600225ca3b in term 0.
I20260809 12:56:25.619316 24956 raft_consensus.cc:2454] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4c009a4dee44df182a202600225ca3b in term 0.
I20260809 12:56:25.619728 24715 leader_election.cc:302] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf96d149d4149e4a0401ae92b0a0d54, a4c009a4dee44df182a202600225ca3b; no voters:
I20260809 12:56:25.620024 25117 raft_consensus.cc:2781] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:56:25.620173 25117 raft_consensus.cc:482] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:56:25.620293 25117 raft_consensus.cc:3037] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:25.622728 25117 raft_consensus.cc:504] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:25.623335 25117 leader_election.cc:288] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [CANDIDATE]: Term 1 election: Requested vote from peers 0bf96d149d4149e4a0401ae92b0a0d54 (127.21.235.65:45555), d4c97719025a45049ba3a2d0e40ab2ff (127.21.235.67:34405)
I20260809 12:56:25.623901 24956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dccbb5300d784157a143a2d83986be2c" candidate_uuid: "a4c009a4dee44df182a202600225ca3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4c97719025a45049ba3a2d0e40ab2ff"
I20260809 12:56:25.623941 24632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dccbb5300d784157a143a2d83986be2c" candidate_uuid: "a4c009a4dee44df182a202600225ca3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf96d149d4149e4a0401ae92b0a0d54"
I20260809 12:56:25.624130 24956 raft_consensus.cc:3037] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:25.624141 24632 raft_consensus.cc:3037] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:25.628006 24632 raft_consensus.cc:2454] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4c009a4dee44df182a202600225ca3b in term 1.
I20260809 12:56:25.628008 24956 raft_consensus.cc:2454] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4c009a4dee44df182a202600225ca3b in term 1.
I20260809 12:56:25.628509 24714 leader_election.cc:302] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4c009a4dee44df182a202600225ca3b, d4c97719025a45049ba3a2d0e40ab2ff; no voters:
I20260809 12:56:25.628927 25117 raft_consensus.cc:2781] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:25.629103 25117 raft_consensus.cc:686] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [term 1 LEADER]: Becoming Leader. State: Replica: a4c009a4dee44df182a202600225ca3b, State: Running, Role: LEADER
I20260809 12:56:25.629454 25117 consensus_queue.cc:237] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:25.632207 24456 catalog_manager.cc:5495] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b reported cstate change: term changed from 0 to 1, leader changed from <none> to a4c009a4dee44df182a202600225ca3b (127.21.235.66). New cstate: current_term: 1 leader_uuid: "a4c009a4dee44df182a202600225ca3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } health_report { overall_health: UNKNOWN } } }
I20260809 12:56:25.653441 25147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:25.648189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40288 (local address 127.21.235.66:32835)
0809 12:56:25.648835 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:25.648839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:25.648857 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:56:25.648907 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:25.648910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:25.648970 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:25.649050 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:56:25.649409 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:25.650304 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:25.651662 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:25.652145 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:25.652242 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:25.652728 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:56:25.652833 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:56:25.652905 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:25.653026 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 12:56:25.653083 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":416,"threads_started":1}
I20260809 12:56:25.661837 24632 raft_consensus.cc:1261] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Refusing update from remote peer a4c009a4dee44df182a202600225ca3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:25.661844 24956 raft_consensus.cc:1261] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff [term 1 FOLLOWER]: Refusing update from remote peer a4c009a4dee44df182a202600225ca3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:25.662456 25096 consensus_queue.cc:1035] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:56:25.663007 25117 consensus_queue.cc:1035] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:56:25.710722 24455 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:56:25.710933 24455 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:56:25.713891 24455 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96: Sending DeleteTablet for 3 replicas of tablet dccbb5300d784157a143a2d83986be2c
I20260809 12:56:25.714861 24928 tablet_service.cc:1514] Processing DeleteTablet for tablet dccbb5300d784157a143a2d83986be2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:25 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:40828
I20260809 12:56:25.714869 24608 tablet_service.cc:1514] Processing DeleteTablet for tablet dccbb5300d784157a143a2d83986be2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:25 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:45952
I20260809 12:56:25.714876 24769 tablet_service.cc:1514] Processing DeleteTablet for tablet dccbb5300d784157a143a2d83986be2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:25 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:40266
I20260809 12:56:25.715137 25123 tablet_replica.cc:330] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54: stopping tablet replica
I20260809 12:56:25.715137 25125 tablet_replica.cc:330] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b: stopping tablet replica
I20260809 12:56:25.715137 25126 tablet_replica.cc:330] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff: stopping tablet replica
I20260809 12:56:25.715328 25125 raft_consensus.cc:2229] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:56:25.715328 25123 raft_consensus.cc:2229] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:56:25.715342 25126 raft_consensus.cc:2229] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:56:25.715512 25123 raft_consensus.cc:2258] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:25.715562 25126 raft_consensus.cc:2258] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:25.715631 25125 raft_consensus.cc:2258] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:25.715926 24455 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 12:56:25.716636 25125 ts_tablet_manager.cc:1905] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:25.716636 25123 ts_tablet_manager.cc:1905] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 12:56:25.716665 24455 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:56:25.717098 25126 ts_tablet_manager.cc:1905] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:25.723238 25123 ts_tablet_manager.cc:1918] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:56:25.723403 25123 log.cc:1196] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/dccbb5300d784157a143a2d83986be2c
I20260809 12:56:25.723694 25126 ts_tablet_manager.cc:1918] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:56:25.723850 25123 ts_tablet_manager.cc:1939] T dccbb5300d784157a143a2d83986be2c P 0bf96d149d4149e4a0401ae92b0a0d54: Deleting consensus metadata
I20260809 12:56:25.723860 25126 log.cc:1196] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/dccbb5300d784157a143a2d83986be2c
I20260809 12:56:25.724328 25126 ts_tablet_manager.cc:1939] T dccbb5300d784157a143a2d83986be2c P d4c97719025a45049ba3a2d0e40ab2ff: Deleting consensus metadata
I20260809 12:56:25.724766 24928 tablet_service.cc:1467] Processing CreateTablet for tablet 20e5557d3e3e4cd49b8b833712f0af9a (DEFAULT_TABLE table=test-table [id=13b8c84850554b98a4bb9d7eb11bcd4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:25.725217 24928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20e5557d3e3e4cd49b8b833712f0af9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:25.725359 24439 catalog_manager.cc:4841] TS d4c97719025a45049ba3a2d0e40ab2ff (127.21.235.67:34405): tablet dccbb5300d784157a143a2d83986be2c (table test-table [id=08e4b175f0184e2fab64512ad371defd]) successfully deleted
I20260809 12:56:25.725493 24769 tablet_service.cc:1467] Processing CreateTablet for tablet 20e5557d3e3e4cd49b8b833712f0af9a (DEFAULT_TABLE table=test-table [id=13b8c84850554b98a4bb9d7eb11bcd4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:25.725981 24769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20e5557d3e3e4cd49b8b833712f0af9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:25.726186 24608 tablet_service.cc:1467] Processing CreateTablet for tablet 20e5557d3e3e4cd49b8b833712f0af9a (DEFAULT_TABLE table=test-table [id=13b8c84850554b98a4bb9d7eb11bcd4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:25.726518 25125 ts_tablet_manager.cc:1918] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:56:25.726606 24608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20e5557d3e3e4cd49b8b833712f0af9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:25.726783 25125 log.cc:1196] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/dccbb5300d784157a143a2d83986be2c
I20260809 12:56:25.726920 24440 catalog_manager.cc:4841] TS 0bf96d149d4149e4a0401ae92b0a0d54 (127.21.235.65:45555): tablet dccbb5300d784157a143a2d83986be2c (table test-table [id=08e4b175f0184e2fab64512ad371defd]) successfully deleted
I20260809 12:56:25.727438 25125 ts_tablet_manager.cc:1939] T dccbb5300d784157a143a2d83986be2c P a4c009a4dee44df182a202600225ca3b: Deleting consensus metadata
I20260809 12:56:25.728543 24441 catalog_manager.cc:4841] TS a4c009a4dee44df182a202600225ca3b (127.21.235.66:32835): tablet dccbb5300d784157a143a2d83986be2c (table test-table [id=08e4b175f0184e2fab64512ad371defd]) successfully deleted
I20260809 12:56:25.732504 25140 tablet_bootstrap.cc:492] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b: Bootstrap starting.
I20260809 12:56:25.732712 25138 tablet_bootstrap.cc:492] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff: Bootstrap starting.
I20260809 12:56:25.734272 25138 tablet_bootstrap.cc:654] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:25.734300 25140 tablet_bootstrap.cc:654] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:25.735157 25139 tablet_bootstrap.cc:492] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54: Bootstrap starting.
I20260809 12:56:25.736931 25140 tablet_bootstrap.cc:492] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b: No bootstrap required, opened a new log
I20260809 12:56:25.737059 25139 tablet_bootstrap.cc:654] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:25.737066 25138 tablet_bootstrap.cc:492] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff: No bootstrap required, opened a new log
I20260809 12:56:25.737134 25140 ts_tablet_manager.cc:1397] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 12:56:25.737263 25138 ts_tablet_manager.cc:1397] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 12:56:25.738031 25140 raft_consensus.cc:348] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:25.738045 25138 raft_consensus.cc:348] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:25.738304 25140 raft_consensus.cc:374] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:25.738452 25140 raft_consensus.cc:729] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4c009a4dee44df182a202600225ca3b, State: Initialized, Role: FOLLOWER
I20260809 12:56:25.738730 25140 consensus_queue.cc:260] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:25.739307 25140 ts_tablet_manager.cc:1428] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 12:56:25.739451 25138 raft_consensus.cc:374] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:25.739558 25138 raft_consensus.cc:729] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4c97719025a45049ba3a2d0e40ab2ff, State: Initialized, Role: FOLLOWER
I20260809 12:56:25.740088 25138 consensus_queue.cc:260] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:25.740705 25138 ts_tablet_manager.cc:1428] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 12:56:25.740952 25139 tablet_bootstrap.cc:492] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54: No bootstrap required, opened a new log
I20260809 12:56:25.741122 25139 ts_tablet_manager.cc:1397] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260809 12:56:25.741904 25139 raft_consensus.cc:348] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:25.742189 25139 raft_consensus.cc:374] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:25.742282 25139 raft_consensus.cc:729] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf96d149d4149e4a0401ae92b0a0d54, State: Initialized, Role: FOLLOWER
I20260809 12:56:25.742697 25139 consensus_queue.cc:260] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:25.743283 25139 ts_tablet_manager.cc:1428] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 12:56:25.848361 25117 raft_consensus.cc:482] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:56:25.848608 25117 raft_consensus.cc:504] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:25.849323 25117 leader_election.cc:288] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bf96d149d4149e4a0401ae92b0a0d54 (127.21.235.65:45555), d4c97719025a45049ba3a2d0e40ab2ff (127.21.235.67:34405)
I20260809 12:56:25.849851 24632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20e5557d3e3e4cd49b8b833712f0af9a" candidate_uuid: "a4c009a4dee44df182a202600225ca3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" is_pre_election: true
I20260809 12:56:25.849978 24956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20e5557d3e3e4cd49b8b833712f0af9a" candidate_uuid: "a4c009a4dee44df182a202600225ca3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" is_pre_election: true
I20260809 12:56:25.850153 24632 raft_consensus.cc:2454] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4c009a4dee44df182a202600225ca3b in term 0.
I20260809 12:56:25.850337 24956 raft_consensus.cc:2454] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4c009a4dee44df182a202600225ca3b in term 0.
I20260809 12:56:25.850636 24715 leader_election.cc:302] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bf96d149d4149e4a0401ae92b0a0d54, a4c009a4dee44df182a202600225ca3b; no voters:
I20260809 12:56:25.850900 25117 raft_consensus.cc:2781] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:56:25.851027 25117 raft_consensus.cc:482] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:56:25.851099 25117 raft_consensus.cc:3037] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:25.853235 25117 raft_consensus.cc:504] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:25.853856 25117 leader_election.cc:288] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [CANDIDATE]: Term 1 election: Requested vote from peers 0bf96d149d4149e4a0401ae92b0a0d54 (127.21.235.65:45555), d4c97719025a45049ba3a2d0e40ab2ff (127.21.235.67:34405)
I20260809 12:56:25.854295 24632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20e5557d3e3e4cd49b8b833712f0af9a" candidate_uuid: "a4c009a4dee44df182a202600225ca3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bf96d149d4149e4a0401ae92b0a0d54"
I20260809 12:56:25.854449 24956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20e5557d3e3e4cd49b8b833712f0af9a" candidate_uuid: "a4c009a4dee44df182a202600225ca3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4c97719025a45049ba3a2d0e40ab2ff"
I20260809 12:56:25.854550 24632 raft_consensus.cc:3037] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:25.854738 24956 raft_consensus.cc:3037] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:25.856778 24956 raft_consensus.cc:2454] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4c009a4dee44df182a202600225ca3b in term 1.
I20260809 12:56:25.857033 24632 raft_consensus.cc:2454] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4c009a4dee44df182a202600225ca3b in term 1.
I20260809 12:56:25.857214 24714 leader_election.cc:302] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4c009a4dee44df182a202600225ca3b, d4c97719025a45049ba3a2d0e40ab2ff; no voters:
I20260809 12:56:25.857491 25117 raft_consensus.cc:2781] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:25.857651 25117 raft_consensus.cc:686] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [term 1 LEADER]: Becoming Leader. State: Replica: a4c009a4dee44df182a202600225ca3b, State: Running, Role: LEADER
I20260809 12:56:25.857925 25117 consensus_queue.cc:237] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } }
I20260809 12:56:25.860996 24455 catalog_manager.cc:5495] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b reported cstate change: term changed from 0 to 1, leader changed from <none> to a4c009a4dee44df182a202600225ca3b (127.21.235.66). New cstate: current_term: 1 leader_uuid: "a4c009a4dee44df182a202600225ca3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4c009a4dee44df182a202600225ca3b" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 32835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 } health_report { overall_health: UNKNOWN } } }
I20260809 12:56:25.899798 24956 raft_consensus.cc:1261] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff [term 1 FOLLOWER]: Refusing update from remote peer a4c009a4dee44df182a202600225ca3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:25.900033 24632 raft_consensus.cc:1261] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Refusing update from remote peer a4c009a4dee44df182a202600225ca3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:25.900570 25096 consensus_queue.cc:1035] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bf96d149d4149e4a0401ae92b0a0d54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:56:25.900866 25117 consensus_queue.cc:1035] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4c97719025a45049ba3a2d0e40ab2ff" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 12:56:25.923177 25101 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 12:56:27.502981 24455 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:56:27.503283 24455 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:56:27.508179 24455 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db46cef39c2c4782aa9c9d2ece608b96: Sending DeleteTablet for 3 replicas of tablet 20e5557d3e3e4cd49b8b833712f0af9a
I20260809 12:56:27.508965 24608 tablet_service.cc:1514] Processing DeleteTablet for tablet 20e5557d3e3e4cd49b8b833712f0af9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:27 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:45952
I20260809 12:56:27.509001 24769 tablet_service.cc:1514] Processing DeleteTablet for tablet 20e5557d3e3e4cd49b8b833712f0af9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:27 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:40266
I20260809 12:56:27.509126 24928 tablet_service.cc:1514] Processing DeleteTablet for tablet 20e5557d3e3e4cd49b8b833712f0af9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:27 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:40828
I20260809 12:56:27.509755 25188 tablet_replica.cc:330] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b: stopping tablet replica
I20260809 12:56:27.510072 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 24536
I20260809 12:56:27.510072 25187 tablet_replica.cc:330] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54: stopping tablet replica
I20260809 12:56:27.510069 25188 raft_consensus.cc:2229] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:56:27.510327 25187 raft_consensus.cc:2229] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:56:27.510713 25187 raft_consensus.cc:2258] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:27.510907 25188 raft_consensus.cc:2258] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:27.511458 25189 tablet_replica.cc:330] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff: stopping tablet replica
I20260809 12:56:27.511662 25189 raft_consensus.cc:2229] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:56:27.511934 25189 raft_consensus.cc:2258] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:27.511950 25187 ts_tablet_manager.cc:1905] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:27.512734 25188 ts_tablet_manager.cc:1905] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:27.512897 25189 ts_tablet_manager.cc:1905] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:27.519133 25187 ts_tablet_manager.cc:1918] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 12:56:27.519269 25187 log.cc:1196] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/20e5557d3e3e4cd49b8b833712f0af9a
I20260809 12:56:27.519757 25187 ts_tablet_manager.cc:1939] T 20e5557d3e3e4cd49b8b833712f0af9a P 0bf96d149d4149e4a0401ae92b0a0d54: Deleting consensus metadata
I20260809 12:56:27.519745 25189 ts_tablet_manager.cc:1918] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 12:56:27.519868 25189 log.cc:1196] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/20e5557d3e3e4cd49b8b833712f0af9a
I20260809 12:56:27.520344 25189 ts_tablet_manager.cc:1939] T 20e5557d3e3e4cd49b8b833712f0af9a P d4c97719025a45049ba3a2d0e40ab2ff: Deleting consensus metadata
I20260809 12:56:27.520951 24440 catalog_manager.cc:4841] TS 0bf96d149d4149e4a0401ae92b0a0d54 (127.21.235.65:45555): tablet 20e5557d3e3e4cd49b8b833712f0af9a (table test-table [id=13b8c84850554b98a4bb9d7eb11bcd4a]) successfully deleted
I20260809 12:56:27.521327 25188 ts_tablet_manager.cc:1918] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 12:56:27.521397 24439 catalog_manager.cc:4841] TS d4c97719025a45049ba3a2d0e40ab2ff (127.21.235.67:34405): tablet 20e5557d3e3e4cd49b8b833712f0af9a (table test-table [id=13b8c84850554b98a4bb9d7eb11bcd4a]) successfully deleted
I20260809 12:56:27.521629 25188 log.cc:1196] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/20e5557d3e3e4cd49b8b833712f0af9a
I20260809 12:56:27.522053 25188 ts_tablet_manager.cc:1939] T 20e5557d3e3e4cd49b8b833712f0af9a P a4c009a4dee44df182a202600225ca3b: Deleting consensus metadata
I20260809 12:56:27.523154 24441 catalog_manager.cc:4841] TS a4c009a4dee44df182a202600225ca3b (127.21.235.66:32835): tablet 20e5557d3e3e4cd49b8b833712f0af9a (table test-table [id=13b8c84850554b98a4bb9d7eb11bcd4a]) successfully deleted
Aug 09 12:56:27 dist-test-slave-gp6n krb5kdc[24406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280185, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 12:56:27.542872 25191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:27.512869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45964 (local address 127.21.235.65:45555)
0809 12:56:27.521891 (+ 9022us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:27.521896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:27.521917 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:56:27.521959 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:27.521962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:27.522018 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:27.522102 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:27.523680 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:27.524487 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:27.525653 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:27.526135 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:27.526231 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:27.536694 (+ 10463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:27.536711 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:27.536729 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:27.536771 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:27.540086 (+ 3315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:27.540780 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:27.540799 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:27.540802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:27.540964 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:27.541284 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:27.541288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:27.541292 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:27.541699 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:27.541977 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:27.542147 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 12:56:27.542334 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8881,"thread_start_us":8731,"threads_started":1}
W20260809 12:56:27.543666 24685 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45964
W20260809 12:56:27.544480 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 24536: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:56:27.544746 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 24536
I20260809 12:56:27.569301 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 24695
Aug 09 12:56:27 dist-test-slave-gp6n krb5kdc[24406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280185, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.235.66@KRBTEST.COM
I20260809 12:56:27.585659 25193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:27.570142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40290 (local address 127.21.235.66:32835)
0809 12:56:27.570858 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:27.570863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:27.570881 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:56:27.570941 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:27.570945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:27.571009 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:27.571111 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:27.572457 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:27.573450 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:27.574786 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:27.575265 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:27.575387 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:27.580086 (+ 4699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:27.580101 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:27.580117 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:27.580153 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:27.583128 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:27.583798 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:27.583805 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:27.583807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:27.583926 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:27.584266 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:27.584269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:27.584271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:27.584623 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:27.584847 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:27.585104 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 12:56:27.585270 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":456,"threads_started":1}
W20260809 12:56:27.586380 24844 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40290
W20260809 12:56:27.587146 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 24695: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:56:27.587420 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 24695
I20260809 12:56:27.614737 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 24856
Aug 09 12:56:27 dist-test-slave-gp6n krb5kdc[24406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280185, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.235.67@KRBTEST.COM
I20260809 12:56:27.630198 25195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:27.615552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40844 (local address 127.21.235.67:34405)
0809 12:56:27.616134 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:27.616139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:27.616155 (+ 16us) server_negotiation.cc:407] Connection header received
0809 12:56:27.616209 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:27.616213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:27.616272 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:27.616364 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:27.617706 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:27.618556 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:27.619801 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:27.620218 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:27.620310 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:27.624750 (+ 4440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:27.624766 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:27.624784 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:27.624822 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:27.627772 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:27.628356 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:27.628362 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:27.628365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:27.628508 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:27.628832 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:27.628836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:27.628837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:27.629188 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:27.629410 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:27.629658 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 12:56:27.629826 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":317,"threads_started":1}
W20260809 12:56:27.630923 25001 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40844
W20260809 12:56:27.631538 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 24856: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:56:27.631690 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 24856
I20260809 12:56:27.656471 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 24421
I20260809 12:56:27.657640 24500 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:27.778667 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 24421
2026-08-09T12:56:27Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8491 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:56:27 dist-test-slave-gp6n krb5kdc[25199](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:56:27 dist-test-slave-gp6n krb5kdc[25199](info): set up 2 sockets
Aug 09 12:56:27 dist-test-slave-gp6n krb5kdc[25199](info): commencing operation
krb5kdc: starting...
W20260809 12:56:29.011897 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:56:29 dist-test-slave-gp6n krb5kdc[25199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:56:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:56:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 12:56:29.222990 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:42109
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:42109
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:29.549489 25214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:29.549768 25214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:29.549934 25214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:29.556979 25214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:56:29.557106 25214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:29.557195 25214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:29.557296 25214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:56:29.557372 25214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:56:29.566931 25214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:42109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:42109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:29.568186 25214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:29.569507 25214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:29.581620 25221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:29.581516 25222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:29.581490 25224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:29.582810 25214 server_base.cc:1034] running on GCE node
I20260809 12:56:29.583568 25214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:29.585013 25214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:29.586218 25214 hybrid_clock.cc:648] HybridClock initialized: now 1786280189586183 us; error 41 us; skew 500 ppm
I20260809 12:56:29.586613 25214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:29 dist-test-slave-gp6n krb5kdc[25199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280189, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:29.589890 25214 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:56:29.591439 25214 webserver.cc:460] Webserver started at http://127.21.235.126:35517/ using document root <none> and password file <none>
I20260809 12:56:29.592154 25214 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:29.592286 25214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:29.592669 25214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:29.595728 25214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "a9abede198e04fce9f665c1c9c698f1a"
format_stamp: "Formatted at 2026-08-09 12:56:29 on dist-test-slave-gp6n"
server_key: "17470a0133ff60efe220c8a264996a7f"
server_key_iv: "6250e02bdb2b86737fb00d75481ae61f"
server_key_version: "encryptionkey@0"
I20260809 12:56:29.596517 25214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "a9abede198e04fce9f665c1c9c698f1a"
format_stamp: "Formatted at 2026-08-09 12:56:29 on dist-test-slave-gp6n"
server_key: "17470a0133ff60efe220c8a264996a7f"
server_key_iv: "6250e02bdb2b86737fb00d75481ae61f"
server_key_version: "encryptionkey@0"
I20260809 12:56:29.601665 25214 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:56:29.605062 25231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:29.606360 25214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:56:29.606608 25214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "a9abede198e04fce9f665c1c9c698f1a"
format_stamp: "Formatted at 2026-08-09 12:56:29 on dist-test-slave-gp6n"
server_key: "17470a0133ff60efe220c8a264996a7f"
server_key_iv: "6250e02bdb2b86737fb00d75481ae61f"
server_key_version: "encryptionkey@0"
I20260809 12:56:29.606822 25214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:29.642946 25214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:29.647610 25214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:29.647912 25214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:29.666960 25214 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:42109
I20260809 12:56:29.667004 25293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:42109 every 8 connection(s)
I20260809 12:56:29.668388 25214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:56:29.672109 25295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:29.672636 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 25214
I20260809 12:56:29.672871 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:56:29.673503 22445 external_mini_cluster.cc:1428] Setting key 3d6d202b19d54ac5c80ae2884eb34055
I20260809 12:56:29.686326 25295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a: Bootstrap starting.
Aug 09 12:56:29 dist-test-slave-gp6n krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:29.689867 25295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:29.691113 25295 log.cc:824] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:29.694228 25295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a: No bootstrap required, opened a new log
I20260809 12:56:29.696568 25298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:29.675611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47320 (local address 127.21.235.126:42109)
0809 12:56:29.676439 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:29.676455 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:29.676505 (+ 50us) server_negotiation.cc:407] Connection header received
0809 12:56:29.678296 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:29.678327 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:29.678850 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:29.679277 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:29.680572 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:29.682510 (+ 1938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:29.683691 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:29.684802 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:29.684968 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:29.689310 (+ 4342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:29.689318 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:29.689332 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:29.689366 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:29.692492 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:29.693255 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:29.693261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:29.693266 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:29.693423 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:29.693840 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:29.693843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:29.693845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:29.694445 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:29.694675 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:29.695169 (+ 494us) server_negotiation.cc:299] Negotiation successful
0809 12:56:29.695510 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":320,"threads_started":1}
I20260809 12:56:29.702347 25295 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9abede198e04fce9f665c1c9c698f1a" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42109 } }
I20260809 12:56:29.702831 25295 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:29.702929 25295 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9abede198e04fce9f665c1c9c698f1a, State: Initialized, Role: FOLLOWER
I20260809 12:56:29.703465 25295 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9abede198e04fce9f665c1c9c698f1a" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42109 } }
I20260809 12:56:29.703676 25295 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:56:29.703768 25295 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:56:29.703891 25295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:29.705936 25295 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9abede198e04fce9f665c1c9c698f1a" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42109 } }
I20260809 12:56:29.706394 25295 leader_election.cc:302] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9abede198e04fce9f665c1c9c698f1a; no voters:
I20260809 12:56:29.707037 25295 leader_election.cc:288] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:56:29.707176 25300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:29.707913 25300 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a [term 1 LEADER]: Becoming Leader. State: Replica: a9abede198e04fce9f665c1c9c698f1a, State: Running, Role: LEADER
I20260809 12:56:29.708395 25300 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9abede198e04fce9f665c1c9c698f1a" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42109 } }
I20260809 12:56:29.709204 25295 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:56:29.711529 25302 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9abede198e04fce9f665c1c9c698f1a. Latest consensus state: current_term: 1 leader_uuid: "a9abede198e04fce9f665c1c9c698f1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9abede198e04fce9f665c1c9c698f1a" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42109 } } }
I20260809 12:56:29.711980 25302 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:29.712388 25301 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9abede198e04fce9f665c1c9c698f1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9abede198e04fce9f665c1c9c698f1a" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42109 } } }
I20260809 12:56:29.712813 25301 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:29.714272 25310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:56:29.719400 25310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:56:29.727507 25310 catalog_manager.cc:1347] Generated new cluster ID: af47943c374849a5b8ae077a4c620ad8
I20260809 12:56:29.727614 25310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:56:29.743259 25310 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:56:29.745185 25310 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:56:29.757256 25310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9abede198e04fce9f665c1c9c698f1a: Generated new TSK 0
I20260809 12:56:29.758327 25310 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
I20260809 12:56:29.813881 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:42109
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:30.164050 25324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:30.164341 25324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:30.164436 25324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:30.171614 25324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:30.171722 25324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:30.171835 25324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:56:30.181628 25324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:42109
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:30.182979 25324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:30.184235 25324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:30.197851 25330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:30.198024 25331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:30.198649 25334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:30.199339 25324 server_base.cc:1034] running on GCE node
I20260809 12:56:30.200037 25324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:30.201040 25324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:30.202277 25324 hybrid_clock.cc:648] HybridClock initialized: now 1786280190202244 us; error 56 us; skew 500 ppm
I20260809 12:56:30.202632 25324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:30 dist-test-slave-gp6n krb5kdc[25199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280190, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:30.207283 25324 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 12:56:30.208916 25324 webserver.cc:460] Webserver started at http://127.21.235.65:34035/ using document root <none> and password file <none>
I20260809 12:56:30.209729 25324 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:30.209903 25324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:30.210340 25324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:30.213708 25324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "c1038481414d4f78b9c608f64542b1ee"
format_stamp: "Formatted at 2026-08-09 12:56:30 on dist-test-slave-gp6n"
server_key: "706509088f022a64c5d6da5f0ca3a379"
server_key_iv: "3c36dbdb5fb7fef9ec53c3b65bd78b57"
server_key_version: "encryptionkey@0"
I20260809 12:56:30.214717 25324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "c1038481414d4f78b9c608f64542b1ee"
format_stamp: "Formatted at 2026-08-09 12:56:30 on dist-test-slave-gp6n"
server_key: "706509088f022a64c5d6da5f0ca3a379"
server_key_iv: "3c36dbdb5fb7fef9ec53c3b65bd78b57"
server_key_version: "encryptionkey@0"
I20260809 12:56:30.222244 25324 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260809 12:56:30.226675 25343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:30.228101 25324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:56:30.228307 25324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "c1038481414d4f78b9c608f64542b1ee"
format_stamp: "Formatted at 2026-08-09 12:56:30 on dist-test-slave-gp6n"
server_key: "706509088f022a64c5d6da5f0ca3a379"
server_key_iv: "3c36dbdb5fb7fef9ec53c3b65bd78b57"
server_key_version: "encryptionkey@0"
I20260809 12:56:30.228540 25324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:30.251112 25324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:30.256032 25324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:30.256455 25324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:30.257483 25324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:30.258939 25324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:30.259018 25324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:30.259125 25324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:30.259176 25324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:30.296060 25324 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:45761
I20260809 12:56:30.296114 25469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:45761 every 8 connection(s)
I20260809 12:56:30.297515 25324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:56:30.305400 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 25324
I20260809 12:56:30.305648 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:56:30.306368 22445 external_mini_cluster.cc:1428] Setting key 5a4f2322a528004eeffcf07526898953
Aug 09 12:56:30 dist-test-slave-gp6n krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280190, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:30.319625 25473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:30.300225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:58323 (local address 127.21.235.126:42109)
0809 12:56:30.300700 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:30.300704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:30.301597 (+ 893us) server_negotiation.cc:407] Connection header received
0809 12:56:30.303490 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:30.303496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:30.303594 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:30.303717 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:30.305214 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:30.306072 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:30.307861 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:30.308299 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:30.308439 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:30.312893 (+ 4454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:30.312898 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:30.312902 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:30.312934 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:30.315899 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:30.316734 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:30.316742 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:30.316748 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:30.316918 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:30.317453 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:30.317461 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:30.317467 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:30.317859 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:30.318024 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:30.318867 (+ 843us) server_negotiation.cc:299] Negotiation successful
0809 12:56:30.319149 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":256,"threads_started":1}
I20260809 12:56:30.320272 25474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:30.300705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42109 (local address 127.21.235.65:58323)
0809 12:56:30.301423 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:30.301471 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:30.303210 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:30.303964 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:30.303973 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:30.304487 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:30.305033 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:30.305051 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:30.306236 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:30.306242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:30.307721 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:30.307733 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:30.308538 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:30.308548 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:30.308778 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:30.309574 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:30.309601 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:30.312659 (+ 3058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:30.316081 (+ 3422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:30.316088 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:30.316102 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:30.316588 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:30.317070 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:30.317073 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:30.317076 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:30.317273 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:30.318019 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:30.318026 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:30.318363 (+ 337us) client_negotiation.cc:769] Sending connection context
0809 12:56:30.318707 (+ 344us) client_negotiation.cc:240] Negotiation successful
0809 12:56:30.319129 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":275,"threads_started":1}
I20260809 12:56:30.321915 25470 heartbeater.cc:347] Connected to a master server at 127.21.235.126:42109
I20260809 12:56:30.322260 25470 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:30.323226 25470 heartbeater.cc:511] Master 127.21.235.126:42109 requested a full tablet report, sending...
I20260809 12:56:30.325582 25251 ts_manager.cc:194] Registered new tserver with Master: c1038481414d4f78b9c608f64542b1ee (127.21.235.65:45761)
I20260809 12:56:30.327143 25251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:58323
WARNING: no policy specified for kudu/127.21.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
I20260809 12:56:30.358656 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:42109
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:30.684603 25477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:30.684867 25477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:30.684962 25477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:30.692189 25477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:30.692293 25477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:30.692404 25477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:56:30.701745 25477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:42109
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:30.702966 25477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:30.704149 25477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:30.717957 25483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:30.717904 25482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:30.718088 25486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:30.718777 25477 server_base.cc:1034] running on GCE node
I20260809 12:56:30.719401 25477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:30.720379 25477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:30.721630 25477 hybrid_clock.cc:648] HybridClock initialized: now 1786280190721580 us; error 72 us; skew 500 ppm
I20260809 12:56:30.721917 25477 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:30 dist-test-slave-gp6n krb5kdc[25199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280190, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:30.725276 25477 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 12:56:30.727185 25477 webserver.cc:460] Webserver started at http://127.21.235.66:34531/ using document root <none> and password file <none>
I20260809 12:56:30.727887 25477 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:30.728032 25477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:30.728434 25477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:30.731807 25477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "c1f8f47884414a29be26b0882b233562"
format_stamp: "Formatted at 2026-08-09 12:56:30 on dist-test-slave-gp6n"
server_key: "9ea523700ef55a616c65e26452acb501"
server_key_iv: "66fd17a59634a3268a8b2819365317f6"
server_key_version: "encryptionkey@0"
I20260809 12:56:30.732700 25477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "c1f8f47884414a29be26b0882b233562"
format_stamp: "Formatted at 2026-08-09 12:56:30 on dist-test-slave-gp6n"
server_key: "9ea523700ef55a616c65e26452acb501"
server_key_iv: "66fd17a59634a3268a8b2819365317f6"
server_key_version: "encryptionkey@0"
I20260809 12:56:30.738176 25477 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:56:30.741555 25494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:30.743078 25477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 12:56:30.743221 25477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "c1f8f47884414a29be26b0882b233562"
format_stamp: "Formatted at 2026-08-09 12:56:30 on dist-test-slave-gp6n"
server_key: "9ea523700ef55a616c65e26452acb501"
server_key_iv: "66fd17a59634a3268a8b2819365317f6"
server_key_version: "encryptionkey@0"
I20260809 12:56:30.743408 25477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:30.755460 25477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:30.759536 25477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:30.759773 25477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:30.760614 25477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:30.761821 25477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:30.761891 25477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:30.761973 25477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:30.762018 25477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:30.795377 25477 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:35801
I20260809 12:56:30.795430 25619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:35801 every 8 connection(s)
I20260809 12:56:30.796715 25477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:56:30.798367 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 25477
I20260809 12:56:30.798604 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:56:30.799127 22445 external_mini_cluster.cc:1428] Setting key b48f095a24df704b464fc84e78869f2b
Aug 09 12:56:30 dist-test-slave-gp6n krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280190, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:30.818344 25473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:30.799608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:38187 (local address 127.21.235.126:42109)
0809 12:56:30.799833 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:30.799840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:30.801243 (+ 1403us) server_negotiation.cc:407] Connection header received
0809 12:56:30.803240 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:30.803248 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:30.803340 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:30.803464 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:30.804868 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:30.806000 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:30.807642 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:30.808155 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:30.808299 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:30.812516 (+ 4217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:30.812521 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:30.812525 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:30.812555 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:30.815349 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:30.816097 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:30.816107 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:30.816110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:30.816231 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:30.816666 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:30.816671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:30.816674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:30.816945 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:30.817099 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:30.817770 (+ 671us) server_negotiation.cc:299] Negotiation successful
0809 12:56:30.818005 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 12:56:30.819283 25623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:30.800216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42109 (local address 127.21.235.66:38187)
0809 12:56:30.801084 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:30.801135 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:30.802972 (+ 1837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:30.803638 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:30.803646 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:30.804150 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:30.804694 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:30.804710 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:30.806177 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:30.806181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:30.807490 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:30.807507 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:30.808392 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:30.808400 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:30.808629 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:30.809353 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:30.809387 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:30.812273 (+ 2886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:30.815487 (+ 3214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:30.815495 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:30.815510 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:30.815972 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:30.816349 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:30.816353 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:30.816357 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:30.816545 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:30.817072 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:30.817080 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:30.817412 (+ 332us) client_negotiation.cc:769] Sending connection context
0809 12:56:30.817662 (+ 250us) client_negotiation.cc:240] Negotiation successful
0809 12:56:30.818164 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":355,"threads_started":1}
WARNING: no policy specified for kudu/127.21.235.67@KRBTEST.COM; defaulting to no policy
I20260809 12:56:30.820859 25621 heartbeater.cc:347] Connected to a master server at 127.21.235.126:42109
I20260809 12:56:30.821216 25621 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:30.821874 25621 heartbeater.cc:511] Master 127.21.235.126:42109 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.67@KRBTEST.COM" created.
I20260809 12:56:30.823390 25251 ts_manager.cc:194] Registered new tserver with Master: c1f8f47884414a29be26b0882b233562 (127.21.235.66:35801)
I20260809 12:56:30.824211 25251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:38187
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
I20260809 12:56:30.849828 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:42109
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:31.207347 25627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:31.207814 25627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:31.208017 25627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:31.216295 25627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:31.216394 25627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:31.216578 25627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:56:31.226513 25627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:42109
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:31.227777 25627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:31.229151 25627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:31.242801 25635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:31.242844 25636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:31.243372 25638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:31.243844 25627 server_base.cc:1034] running on GCE node
I20260809 12:56:31.244432 25627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:31.245350 25627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:31.246527 25627 hybrid_clock.cc:648] HybridClock initialized: now 1786280191246473 us; error 34 us; skew 500 ppm
I20260809 12:56:31.246966 25627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:31 dist-test-slave-gp6n krb5kdc[25199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280191, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:31.250783 25627 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 12:56:31.252341 25627 webserver.cc:460] Webserver started at http://127.21.235.67:38235/ using document root <none> and password file <none>
I20260809 12:56:31.253086 25627 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:31.253243 25627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:31.253597 25627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:31.256825 25627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "d000bc71f7d2479580b71ee223cf5deb"
format_stamp: "Formatted at 2026-08-09 12:56:31 on dist-test-slave-gp6n"
server_key: "d02430456f48b13bf519d64cd1c81021"
server_key_iv: "f95944dfd4157db7e95e765254ae520b"
server_key_version: "encryptionkey@0"
I20260809 12:56:31.257685 25627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "d000bc71f7d2479580b71ee223cf5deb"
format_stamp: "Formatted at 2026-08-09 12:56:31 on dist-test-slave-gp6n"
server_key: "d02430456f48b13bf519d64cd1c81021"
server_key_iv: "f95944dfd4157db7e95e765254ae520b"
server_key_version: "encryptionkey@0"
I20260809 12:56:31.263150 25627 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260809 12:56:31.266897 25647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:31.268217 25627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 12:56:31.268467 25627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "d000bc71f7d2479580b71ee223cf5deb"
format_stamp: "Formatted at 2026-08-09 12:56:31 on dist-test-slave-gp6n"
server_key: "d02430456f48b13bf519d64cd1c81021"
server_key_iv: "f95944dfd4157db7e95e765254ae520b"
server_key_version: "encryptionkey@0"
I20260809 12:56:31.268687 25627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:31.284250 25627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:31.288982 25627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:31.289314 25627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:31.290267 25627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:31.291769 25627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:31.291886 25627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:31.292033 25627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:31.292112 25627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:31.326139 25627 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:44489
I20260809 12:56:31.326232 25773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:44489 every 8 connection(s)
I20260809 12:56:31.327450 25627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:56:31.329789 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 25627
I20260809 12:56:31.330013 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:56:31.330667 22445 external_mini_cluster.cc:1428] Setting key fa0e1a6f45629b11df33fc66fbe23a0b
I20260809 12:56:31.330844 25470 heartbeater.cc:503] Master 127.21.235.126:42109 was elected leader, sending a full tablet report...
Aug 09 12:56:31 dist-test-slave-gp6n krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280191, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:31.348500 25776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:31.330189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:54807 (local address 127.21.235.126:42109)
0809 12:56:31.330805 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:31.330812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:31.331851 (+ 1039us) server_negotiation.cc:407] Connection header received
0809 12:56:31.334198 (+ 2347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:31.334202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:31.334262 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:31.334349 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:31.335860 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:31.336660 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:31.338160 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:31.338639 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:31.338735 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:31.342057 (+ 3322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:31.342062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:31.342065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:31.342094 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:31.345016 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:31.345998 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:31.346002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:31.346005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:31.346121 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:31.346711 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:31.346715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:31.346717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:31.347032 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:31.347196 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:31.347939 (+ 743us) server_negotiation.cc:299] Negotiation successful
0809 12:56:31.348168 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":1922,"threads_started":1}
I20260809 12:56:31.349403 25777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:31.330784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42109 (local address 127.21.235.67:54807)
0809 12:56:31.331684 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:31.331735 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:31.333896 (+ 2161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:31.334629 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:31.334637 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:31.335128 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:31.335717 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:31.335734 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:31.336800 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:31.336806 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:31.338037 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:31.338049 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:31.338831 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:31.338836 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:31.339030 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:31.339688 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:31.339710 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:31.341869 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:31.345180 (+ 3311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:31.345187 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:31.345220 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:31.345850 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:31.346265 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:31.346270 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:31.346274 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:31.346596 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:31.347212 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:31.347222 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:31.347576 (+ 354us) client_negotiation.cc:769] Sending connection context
0809 12:56:31.347836 (+ 260us) client_negotiation.cc:240] Negotiation successful
0809 12:56:31.348235 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":349,"threads_started":1}
I20260809 12:56:31.350826 25774 heartbeater.cc:347] Connected to a master server at 127.21.235.126:42109
I20260809 12:56:31.351164 25774 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:31.351927 25774 heartbeater.cc:511] Master 127.21.235.126:42109 requested a full tablet report, sending...
I20260809 12:56:31.353426 25251 ts_manager.cc:194] Registered new tserver with Master: d000bc71f7d2479580b71ee223cf5deb (127.21.235.67:44489)
I20260809 12:56:31.354166 25251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:54807
I20260809 12:56:31.358556 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 12:56:31.392917 25776 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:56:31.390660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34422 (local address 127.21.235.126:42109)
0809 12:56:31.390837 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:31.390842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:31.391261 (+ 419us) server_negotiation.cc:407] Connection header received
0809 12:56:31.391416 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:31.391420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:31.391498 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:31.391594 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:31.392720 (+ 1126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34422: BlockingRecv error: recv got EOF from 127.0.0.1:34422 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 12:56:31.396311 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 25324
W20260809 12:56:31.399046 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 25324: Not authorized: Client connection negotiation failed: client connection to 127.21.235.65:45761: server requires authentication, but client does not have Kerberos credentials available
I20260809 12:56:31.399238 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 25324
I20260809 12:56:31.419116 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 25477
W20260809 12:56:31.421849 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 25477: Not authorized: Client connection negotiation failed: client connection to 127.21.235.66:35801: server requires authentication, but client does not have Kerberos credentials available
I20260809 12:56:31.422035 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 25477
I20260809 12:56:31.440559 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 25627
W20260809 12:56:31.443514 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 25627: Not authorized: Client connection negotiation failed: client connection to 127.21.235.67:44489: server requires authentication, but client does not have Kerberos credentials available
I20260809 12:56:31.443672 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 25627
I20260809 12:56:31.461295 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 25214
I20260809 12:56:31.462625 25286 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:31.566811 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 25214
2026-08-09T12:56:31Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3784 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:56:31 dist-test-slave-gp6n krb5kdc[25794](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:56:31 dist-test-slave-gp6n krb5kdc[25794](info): set up 2 sockets
Aug 09 12:56:31 dist-test-slave-gp6n krb5kdc[25794](info): commencing operation
krb5kdc: starting...
W20260809 12:56:32.787802 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:56:32 dist-test-slave-gp6n krb5kdc[25794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:56:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:56:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 12:56:33.005752 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:32971
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:32971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:33.340580 25808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:33.340857 25808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:33.340999 25808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:33.348738 25808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:56:33.348842 25808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:33.348889 25808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:33.348948 25808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:56:33.348995 25808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:56:33.358709 25808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:32971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:32971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:33.359911 25808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:33.361151 25808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:33.372849 25814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:33.372824 25815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:33.372786 25817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:33.374367 25808 server_base.cc:1034] running on GCE node
I20260809 12:56:33.375129 25808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:33.376490 25808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:33.377691 25808 hybrid_clock.cc:648] HybridClock initialized: now 1786280193377655 us; error 32 us; skew 500 ppm
I20260809 12:56:33.378037 25808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:33 dist-test-slave-gp6n krb5kdc[25794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280193, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:33.385309 25808 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:56:33.386893 25808 webserver.cc:460] Webserver started at http://127.21.235.126:43317/ using document root <none> and password file <none>
I20260809 12:56:33.387562 25808 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:33.387666 25808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:33.387931 25808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:33.391039 25808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "36f126e8c23043dcbe263f4f1745c35f"
format_stamp: "Formatted at 2026-08-09 12:56:33 on dist-test-slave-gp6n"
server_key: "9450d78a05647af58320f60863c98da9"
server_key_iv: "6ab3b1a9b1d86aacbd9e1294b67ffe03"
server_key_version: "encryptionkey@0"
I20260809 12:56:33.391856 25808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "36f126e8c23043dcbe263f4f1745c35f"
format_stamp: "Formatted at 2026-08-09 12:56:33 on dist-test-slave-gp6n"
server_key: "9450d78a05647af58320f60863c98da9"
server_key_iv: "6ab3b1a9b1d86aacbd9e1294b67ffe03"
server_key_version: "encryptionkey@0"
I20260809 12:56:33.397333 25808 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:56:33.400743 25825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:33.402251 25808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:56:33.402477 25808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "36f126e8c23043dcbe263f4f1745c35f"
format_stamp: "Formatted at 2026-08-09 12:56:33 on dist-test-slave-gp6n"
server_key: "9450d78a05647af58320f60863c98da9"
server_key_iv: "6ab3b1a9b1d86aacbd9e1294b67ffe03"
server_key_version: "encryptionkey@0"
I20260809 12:56:33.402689 25808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:33.445185 25808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:33.449383 25808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:33.449662 25808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:33.468253 25808 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:32971
I20260809 12:56:33.468292 25880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:32971 every 8 connection(s)
I20260809 12:56:33.469615 25808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:56:33.473295 25881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:33.477397 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 25808
I20260809 12:56:33.477641 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:56:33.478518 22445 external_mini_cluster.cc:1428] Setting key be7afda02f4e50dfa90adc2249e3a783
I20260809 12:56:33.484956 25881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f: Bootstrap starting.
I20260809 12:56:33.488647 25881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:33.489814 25881 log.cc:824] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:33.493211 25881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f: No bootstrap required, opened a new log
Aug 09 12:56:33 dist-test-slave-gp6n krb5kdc[25794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:33.503785 25884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:33.480395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36454 (local address 127.21.235.126:32971)
0809 12:56:33.481186 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:33.481196 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:33.481237 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:56:33.483445 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:33.483473 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:33.483846 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:33.484152 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:33.486020 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:33.487481 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:33.489292 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:33.490056 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:33.490188 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:33.495027 (+ 4839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:33.495039 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:33.495063 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:33.495114 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:33.499278 (+ 4164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:33.499936 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:33.499950 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:33.499958 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:33.500168 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:33.500544 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:33.500549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:33.500553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:33.501258 (+ 705us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:33.501624 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:33.501927 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 12:56:33.502431 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":390,"threads_started":1}
I20260809 12:56:33.504043 25881 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f126e8c23043dcbe263f4f1745c35f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 32971 } }
I20260809 12:56:33.504444 25881 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:33.504573 25881 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36f126e8c23043dcbe263f4f1745c35f, State: Initialized, Role: FOLLOWER
I20260809 12:56:33.505110 25881 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f126e8c23043dcbe263f4f1745c35f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 32971 } }
I20260809 12:56:33.505371 25881 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:56:33.505520 25881 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:56:33.505720 25881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:33.508029 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:56:33.508662 25881 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f126e8c23043dcbe263f4f1745c35f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 32971 } }
I20260809 12:56:33.509217 25881 leader_election.cc:302] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36f126e8c23043dcbe263f4f1745c35f; no voters:
I20260809 12:56:33.509860 25881 leader_election.cc:288] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:56:33.509984 25886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:33.512068 25881 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:56:33.522585 25886 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f [term 1 LEADER]: Becoming Leader. State: Replica: 36f126e8c23043dcbe263f4f1745c35f, State: Running, Role: LEADER
I20260809 12:56:33.523413 25886 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f126e8c23043dcbe263f4f1745c35f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 32971 } }
I20260809 12:56:33.527371 25887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36f126e8c23043dcbe263f4f1745c35f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f126e8c23043dcbe263f4f1745c35f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 32971 } } }
I20260809 12:56:33.527451 25888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36f126e8c23043dcbe263f4f1745c35f. Latest consensus state: current_term: 1 leader_uuid: "36f126e8c23043dcbe263f4f1745c35f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f126e8c23043dcbe263f4f1745c35f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 32971 } } }
I20260809 12:56:33.527838 25887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:33.527841 25888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:33.528815 25901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:56:33.536454 25901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:56:33.545917 25901 catalog_manager.cc:1347] Generated new cluster ID: 9c55b24889514af786314df04486e70e
I20260809 12:56:33.546097 25901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:56:33.562629 25884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:33.553890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36470 (local address 127.21.235.126:32971)
0809 12:56:33.554142 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:33.554153 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:33.555052 (+ 899us) server_negotiation.cc:407] Connection header received
0809 12:56:33.555297 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:33.555310 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:33.555385 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:33.555505 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:33.556609 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:33.557449 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:33.558930 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:33.559415 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:33.559575 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:33.560797 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:33.560806 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:33.560810 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:33.561572 (+ 762us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:33.561645 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:33.561678 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:33.561837 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:33.562000 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 12:56:33.562112 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":67}
I20260809 12:56:33.585146 25901 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:56:33.586584 25901 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:56:33.596251 25901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36f126e8c23043dcbe263f4f1745c35f: Generated new TSK 0
I20260809 12:56:33.597122 25901 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:56:33.607278 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 25808
I20260809 12:56:33.608556 25876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:33.699560 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 25808
2026-08-09T12:56:33Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2127 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:56:33 dist-test-slave-gp6n krb5kdc[25916](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:56:33 dist-test-slave-gp6n krb5kdc[25916](info): set up 2 sockets
Aug 09 12:56:33 dist-test-slave-gp6n krb5kdc[25916](info): commencing operation
krb5kdc: starting...
W20260809 12:56:34.880368 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.006s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:56:35 dist-test-slave-gp6n krb5kdc[25916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:56:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:56:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 12:56:35.101876 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:34287
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:34287
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:35.444947 25931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:35.445235 25931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:35.445325 25931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:35.452302 25931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:56:35.452402 25931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:35.452464 25931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:35.452522 25931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:56:35.452565 25931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:56:35.461913 25931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:34287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:34287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:35.463152 25931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:35.464344 25931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:35.476193 25938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:35.476305 25937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:35.476269 25940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:35.476469 25931 server_base.cc:1034] running on GCE node
I20260809 12:56:35.477599 25931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:35.478979 25931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:35.480182 25931 hybrid_clock.cc:648] HybridClock initialized: now 1786280195480120 us; error 47 us; skew 500 ppm
I20260809 12:56:35.480477 25931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:35 dist-test-slave-gp6n krb5kdc[25916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280195, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:35.483645 25931 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:56:35.485111 25931 webserver.cc:460] Webserver started at http://127.21.235.126:33321/ using document root <none> and password file <none>
I20260809 12:56:35.485750 25931 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:35.485852 25931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:35.486110 25931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:35.489108 25931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "68205bbca45a4b6ba5b759c10446ce1e"
format_stamp: "Formatted at 2026-08-09 12:56:35 on dist-test-slave-gp6n"
server_key: "3e5c2b0c3e3547251f8610fb4f992bfd"
server_key_iv: "38e8e9bf14e96a472b80cee02baccfd9"
server_key_version: "encryptionkey@0"
I20260809 12:56:35.489848 25931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "68205bbca45a4b6ba5b759c10446ce1e"
format_stamp: "Formatted at 2026-08-09 12:56:35 on dist-test-slave-gp6n"
server_key: "3e5c2b0c3e3547251f8610fb4f992bfd"
server_key_iv: "38e8e9bf14e96a472b80cee02baccfd9"
server_key_version: "encryptionkey@0"
I20260809 12:56:35.495217 25931 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260809 12:56:35.498721 25948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:35.500206 25931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:56:35.500355 25931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "68205bbca45a4b6ba5b759c10446ce1e"
format_stamp: "Formatted at 2026-08-09 12:56:35 on dist-test-slave-gp6n"
server_key: "3e5c2b0c3e3547251f8610fb4f992bfd"
server_key_iv: "38e8e9bf14e96a472b80cee02baccfd9"
server_key_version: "encryptionkey@0"
I20260809 12:56:35.500540 25931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:35.515645 25931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:35.519714 25931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:35.519955 25931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:35.538785 25931 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:34287
I20260809 12:56:35.538832 26001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:34287 every 8 connection(s)
I20260809 12:56:35.540140 25931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:56:35.542158 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 25931
I20260809 12:56:35.542385 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:56:35.543267 22445 external_mini_cluster.cc:1428] Setting key 14760126141f6d0f35ac3ad165b301d7
I20260809 12:56:35.545269 26002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 12:56:35 dist-test-slave-gp6n krb5kdc[25916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:35.558735 26002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e: Bootstrap starting.
I20260809 12:56:35.562057 26002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:35.563115 26002 log.cc:824] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:35.565864 26005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:35.545322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44114 (local address 127.21.235.126:34287)
0809 12:56:35.546128 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:35.546140 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:35.546180 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:56:35.547814 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:35.547835 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:35.548222 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:35.548556 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:35.550243 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:35.551643 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:35.553363 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:35.554006 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:35.554129 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:35.558730 (+ 4601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:35.558737 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:35.558751 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:35.558789 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:35.561782 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:35.562591 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:35.562598 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:35.562604 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:35.562765 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:35.563154 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:35.563159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:35.563162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:35.563726 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:35.564088 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:35.564541 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 12:56:35.564952 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":348,"threads_started":1}
I20260809 12:56:35.565980 26002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e: No bootstrap required, opened a new log
I20260809 12:56:35.574380 26002 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68205bbca45a4b6ba5b759c10446ce1e" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34287 } }
I20260809 12:56:35.574770 26002 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:35.574872 26002 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68205bbca45a4b6ba5b759c10446ce1e, State: Initialized, Role: FOLLOWER
I20260809 12:56:35.575526 26002 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68205bbca45a4b6ba5b759c10446ce1e" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34287 } }
I20260809 12:56:35.575830 26002 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:56:35.575989 26002 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:56:35.576181 26002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:35.578239 26002 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68205bbca45a4b6ba5b759c10446ce1e" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34287 } }
I20260809 12:56:35.578732 26002 leader_election.cc:302] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68205bbca45a4b6ba5b759c10446ce1e; no voters:
I20260809 12:56:35.579281 26002 leader_election.cc:288] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:56:35.579455 26007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:35.580098 26007 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e [term 1 LEADER]: Becoming Leader. State: Replica: 68205bbca45a4b6ba5b759c10446ce1e, State: Running, Role: LEADER
I20260809 12:56:35.580639 26007 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68205bbca45a4b6ba5b759c10446ce1e" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34287 } }
I20260809 12:56:35.581367 26002 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:56:35.583590 26008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68205bbca45a4b6ba5b759c10446ce1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68205bbca45a4b6ba5b759c10446ce1e" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34287 } } }
I20260809 12:56:35.583563 26009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68205bbca45a4b6ba5b759c10446ce1e. Latest consensus state: current_term: 1 leader_uuid: "68205bbca45a4b6ba5b759c10446ce1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68205bbca45a4b6ba5b759c10446ce1e" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34287 } } }
I20260809 12:56:35.583958 26008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:35.583956 26009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:35.586463 26015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:56:35.591843 26015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:56:35.599926 26015 catalog_manager.cc:1347] Generated new cluster ID: 0b5cb0f0310c4d9ba5f4604a1bd65544
I20260809 12:56:35.600068 26015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:56:35.615624 26015 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:56:35.616891 26015 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:56:35.625465 26015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68205bbca45a4b6ba5b759c10446ce1e: Generated new TSK 0
I20260809 12:56:35.626278 26015 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
I20260809 12:56:35.683813 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:34287
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:36.013212 26034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:36.013500 26034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:36.013597 26034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:36.020673 26034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:36.020785 26034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:36.020907 26034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:56:36.030614 26034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:34287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:36.031939 26034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:36.033212 26034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:36.046715 26039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:36.046765 26041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:36.046789 26043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:36.047551 26034 server_base.cc:1034] running on GCE node
I20260809 12:56:36.048197 26034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:36.049190 26034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:36.050482 26034 hybrid_clock.cc:648] HybridClock initialized: now 1786280196050432 us; error 54 us; skew 500 ppm
I20260809 12:56:36.050800 26034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:36 dist-test-slave-gp6n krb5kdc[25916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280196, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:36.054088 26034 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 12:56:36.055565 26034 webserver.cc:460] Webserver started at http://127.21.235.65:33925/ using document root <none> and password file <none>
I20260809 12:56:36.056252 26034 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:36.056399 26034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:36.056703 26034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:36.059643 26034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "dcf563cc2b74487aa3d737038b20eedc"
format_stamp: "Formatted at 2026-08-09 12:56:36 on dist-test-slave-gp6n"
server_key: "a8149d76e742e1d21d409eefe3c156a0"
server_key_iv: "f61c34d218fb9175ac1598dedd57e17e"
server_key_version: "encryptionkey@0"
I20260809 12:56:36.060447 26034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "dcf563cc2b74487aa3d737038b20eedc"
format_stamp: "Formatted at 2026-08-09 12:56:36 on dist-test-slave-gp6n"
server_key: "a8149d76e742e1d21d409eefe3c156a0"
server_key_iv: "f61c34d218fb9175ac1598dedd57e17e"
server_key_version: "encryptionkey@0"
I20260809 12:56:36.066097 26034 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 12:56:36.069461 26050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:36.070845 26034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:56:36.071039 26034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "dcf563cc2b74487aa3d737038b20eedc"
format_stamp: "Formatted at 2026-08-09 12:56:36 on dist-test-slave-gp6n"
server_key: "a8149d76e742e1d21d409eefe3c156a0"
server_key_iv: "f61c34d218fb9175ac1598dedd57e17e"
server_key_version: "encryptionkey@0"
I20260809 12:56:36.071209 26034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:36.127619 26034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:36.131888 26034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:36.132203 26034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:36.133203 26034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:36.134627 26034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:36.134702 26034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:36.134874 26034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:36.134938 26034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:36.169407 26034 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:35157
I20260809 12:56:36.169623 26176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:35157 every 8 connection(s)
I20260809 12:56:36.170851 26034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:56:36.174274 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 26034
I20260809 12:56:36.174533 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:56:36.175318 22445 external_mini_cluster.cc:1428] Setting key 823eb75ccd68cbf8376ab4c5c9eb7c8a
Aug 09 12:56:36 dist-test-slave-gp6n krb5kdc[25916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280196, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:36.192847 26179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:36.173566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:40549 (local address 127.21.235.126:34287)
0809 12:56:36.174132 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:36.174140 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:36.175195 (+ 1055us) server_negotiation.cc:407] Connection header received
0809 12:56:36.177296 (+ 2101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:36.177300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:36.177405 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:36.177517 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:36.179143 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:36.180291 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:36.182076 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:36.182784 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:36.182959 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:36.186719 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:36.186724 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:36.186727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:36.186784 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:36.189563 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:36.190516 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:36.190520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:36.190522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:36.190642 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:36.191179 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:36.191183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:36.191184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:36.191424 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:36.191532 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:36.192434 (+ 902us) server_negotiation.cc:299] Negotiation successful
0809 12:56:36.192596 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":284,"threads_started":1}
I20260809 12:56:36.194429 26182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:36.174152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:34287 (local address 127.21.235.65:40549)
0809 12:56:36.175013 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:36.175061 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:36.177001 (+ 1940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:36.177741 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:36.177749 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:36.178319 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:36.178964 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:36.178982 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:36.180474 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:36.180477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:36.181906 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:36.181918 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:36.183019 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:36.183023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:36.183221 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:36.183954 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:36.183977 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:36.186518 (+ 2541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:36.189703 (+ 3185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:36.189714 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:36.189731 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:36.190365 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:36.190756 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:36.190764 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:36.190770 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:36.191065 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:36.191550 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:36.191560 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:36.192030 (+ 470us) client_negotiation.cc:769] Sending connection context
0809 12:56:36.192341 (+ 311us) client_negotiation.cc:240] Negotiation successful
0809 12:56:36.192929 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":379,"threads_started":1}
I20260809 12:56:36.196079 26177 heartbeater.cc:347] Connected to a master server at 127.21.235.126:34287
I20260809 12:56:36.196380 26177 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:36.197113 26177 heartbeater.cc:511] Master 127.21.235.126:34287 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.235.66@KRBTEST.COM; defaulting to no policy
I20260809 12:56:36.199360 25965 ts_manager.cc:194] Registered new tserver with Master: dcf563cc2b74487aa3d737038b20eedc (127.21.235.65:35157)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.66@KRBTEST.COM" created.
I20260809 12:56:36.201254 25965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:40549
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
I20260809 12:56:36.227736 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:34287
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:36.563303 26185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:36.563568 26185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:36.563762 26185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:36.571072 26185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:36.571201 26185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:36.571410 26185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:56:36.581235 26185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:34287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:36.582612 26185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:36.583880 26185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:36.596920 26195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:36.597007 26192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:36.597007 26193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:36.598067 26185 server_base.cc:1034] running on GCE node
I20260809 12:56:36.598827 26185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:36.599972 26185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:36.601205 26185 hybrid_clock.cc:648] HybridClock initialized: now 1786280196601144 us; error 54 us; skew 500 ppm
I20260809 12:56:36.601565 26185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:36 dist-test-slave-gp6n krb5kdc[25916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280196, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:36.604768 26185 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 12:56:36.606253 26185 webserver.cc:460] Webserver started at http://127.21.235.66:34563/ using document root <none> and password file <none>
I20260809 12:56:36.607000 26185 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:36.607146 26185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:36.607457 26185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:36.610283 26185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "ac3e9c6da6644f3a8c1259ee5e757741"
format_stamp: "Formatted at 2026-08-09 12:56:36 on dist-test-slave-gp6n"
server_key: "c67a38533d1fe42ba5400466cc592cb5"
server_key_iv: "fcf2d0c34401aaa9e0cd0fd3cb2d40b4"
server_key_version: "encryptionkey@0"
I20260809 12:56:36.611115 26185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "ac3e9c6da6644f3a8c1259ee5e757741"
format_stamp: "Formatted at 2026-08-09 12:56:36 on dist-test-slave-gp6n"
server_key: "c67a38533d1fe42ba5400466cc592cb5"
server_key_iv: "fcf2d0c34401aaa9e0cd0fd3cb2d40b4"
server_key_version: "encryptionkey@0"
I20260809 12:56:36.616062 26185 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:56:36.619484 26203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:36.621021 26185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:56:36.621178 26185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "ac3e9c6da6644f3a8c1259ee5e757741"
format_stamp: "Formatted at 2026-08-09 12:56:36 on dist-test-slave-gp6n"
server_key: "c67a38533d1fe42ba5400466cc592cb5"
server_key_iv: "fcf2d0c34401aaa9e0cd0fd3cb2d40b4"
server_key_version: "encryptionkey@0"
I20260809 12:56:36.621395 26185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:36.635437 26185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:36.639597 26185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:36.639923 26185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:36.640918 26185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:36.642345 26185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:36.642446 26185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260809 12:56:36.642601 26185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:36.642654 26185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:36.678329 26185 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:46123
I20260809 12:56:36.678351 26327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:46123 every 8 connection(s)
I20260809 12:56:36.679798 26185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:56:36.687772 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 26185
I20260809 12:56:36.687953 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:56:36.688532 22445 external_mini_cluster.cc:1428] Setting key ec5012791735ce018f6a2e4ce673069f
Aug 09 12:56:36 dist-test-slave-gp6n krb5kdc[25916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280196, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:36.700558 26179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:36.682554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:46081 (local address 127.21.235.126:34287)
0809 12:56:36.682758 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:36.682763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:36.683903 (+ 1140us) server_negotiation.cc:407] Connection header received
0809 12:56:36.685875 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:36.685880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:36.685946 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:36.686034 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:36.687984 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:36.689058 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:36.690922 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:36.691348 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:36.691484 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:36.694918 (+ 3434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:36.694923 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:36.694926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:36.694964 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:36.697535 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:36.698255 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:36.698267 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:36.698270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:36.698393 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:36.698874 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:36.698879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:36.698883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:36.699149 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:36.699304 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:36.699982 (+ 678us) server_negotiation.cc:299] Negotiation successful
0809 12:56:36.700210 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 12:56:36.701498 26330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:36.682987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:34287 (local address 127.21.235.66:46081)
0809 12:56:36.683720 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:36.683773 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:36.685617 (+ 1844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:36.686374 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:36.686384 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:36.687079 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:36.687753 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:36.687771 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:36.689219 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:36.689231 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:36.690780 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:36.690792 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:36.691592 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:36.691600 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:36.691819 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:36.692494 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:36.692516 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:36.694686 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:36.697684 (+ 2998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:36.697693 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:36.697709 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:36.698123 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:36.698551 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:36.698555 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:36.698559 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:36.698751 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:36.699280 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:36.699287 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:36.699615 (+ 328us) client_negotiation.cc:769] Sending connection context
0809 12:56:36.699866 (+ 251us) client_negotiation.cc:240] Negotiation successful
0809 12:56:36.700379 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":280,"threads_started":1}
I20260809 12:56:36.702998 26328 heartbeater.cc:347] Connected to a master server at 127.21.235.126:34287
I20260809 12:56:36.703369 26328 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:36.704288 26328 heartbeater.cc:511] Master 127.21.235.126:34287 requested a full tablet report, sending...
I20260809 12:56:36.705839 25965 ts_manager.cc:194] Registered new tserver with Master: ac3e9c6da6644f3a8c1259ee5e757741 (127.21.235.66:46123)
I20260809 12:56:36.706736 25965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:46081
WARNING: no policy specified for kudu/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
I20260809 12:56:36.738644 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:34287
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:37.067802 26334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:37.068074 26334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:37.068169 26334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:37.075740 26334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:37.075847 26334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:37.076043 26334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:56:37.086014 26334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:34287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:37.087528 26334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:37.088788 26334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:37.105633 26340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:37.105640 26343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:37.105681 26341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:37.106599 26334 server_base.cc:1034] running on GCE node
I20260809 12:56:37.107227 26334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:37.108232 26334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:37.109442 26334 hybrid_clock.cc:648] HybridClock initialized: now 1786280197109389 us; error 42 us; skew 500 ppm
I20260809 12:56:37.109823 26334 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:37 dist-test-slave-gp6n krb5kdc[25916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280197, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:37.113111 26334 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 12:56:37.114662 26334 webserver.cc:460] Webserver started at http://127.21.235.67:37401/ using document root <none> and password file <none>
I20260809 12:56:37.115360 26334 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:37.115509 26334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:37.115814 26334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:37.118790 26334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "9e4af83874a24deaa29ae9fabb0027c7"
format_stamp: "Formatted at 2026-08-09 12:56:37 on dist-test-slave-gp6n"
server_key: "07db15f1f702124d9525275edd72920f"
server_key_iv: "0890e50cc3df8fb304eeba32386743e0"
server_key_version: "encryptionkey@0"
I20260809 12:56:37.119592 26334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "9e4af83874a24deaa29ae9fabb0027c7"
format_stamp: "Formatted at 2026-08-09 12:56:37 on dist-test-slave-gp6n"
server_key: "07db15f1f702124d9525275edd72920f"
server_key_iv: "0890e50cc3df8fb304eeba32386743e0"
server_key_version: "encryptionkey@0"
I20260809 12:56:37.124761 26334 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 12:56:37.128297 26350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:37.129736 26334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:56:37.129901 26334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "9e4af83874a24deaa29ae9fabb0027c7"
format_stamp: "Formatted at 2026-08-09 12:56:37 on dist-test-slave-gp6n"
server_key: "07db15f1f702124d9525275edd72920f"
server_key_iv: "0890e50cc3df8fb304eeba32386743e0"
server_key_version: "encryptionkey@0"
I20260809 12:56:37.130069 26334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:37.142475 26334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:37.146201 26334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:37.146591 26334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:37.147531 26334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:37.148881 26334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:37.148955 26334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:37.149125 26334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:37.149179 26334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:37.186955 26334 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:36315
I20260809 12:56:37.186980 26477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:36315 every 8 connection(s)
I20260809 12:56:37.188318 26334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:56:37.189248 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 26334
I20260809 12:56:37.189458 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:56:37.189952 22445 external_mini_cluster.cc:1428] Setting key 2df13fdbdd283867bf0f0d74f758b825
Aug 09 12:56:37 dist-test-slave-gp6n krb5kdc[25916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280197, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:37.205166 26177 heartbeater.cc:503] Master 127.21.235.126:34287 was elected leader, sending a full tablet report...
I20260809 12:56:37.209559 26179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:37.192352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:47611 (local address 127.21.235.126:34287)
0809 12:56:37.193088 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:37.193095 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:37.193118 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:56:37.194595 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:37.194600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:37.194687 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:37.194806 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:37.196307 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:37.197071 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:37.198695 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:37.199134 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:37.199276 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:37.202985 (+ 3709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:37.202991 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:37.203012 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:37.203062 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:37.206555 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:37.207560 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:37.207564 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:37.207566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:37.207699 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:37.208090 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:37.208094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:37.208096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:37.208360 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:37.208476 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:37.209128 (+ 652us) server_negotiation.cc:299] Negotiation successful
0809 12:56:37.209307 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611}
I20260809 12:56:37.210641 26482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:37.191669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:34287 (local address 127.21.235.67:47611)
0809 12:56:37.192572 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:37.192628 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:37.194277 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:37.195044 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:37.195052 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:37.195564 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:37.196137 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:37.196154 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:37.197237 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:37.197241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:37.198555 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:37.198567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:37.199367 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:37.199377 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:37.199632 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:37.200354 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:37.200376 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:37.202719 (+ 2343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:37.206732 (+ 4013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:37.206742 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:37.206760 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:37.207439 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:37.207796 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:37.207799 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:37.207801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:37.207990 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:37.208473 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:37.208478 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:37.208797 (+ 319us) client_negotiation.cc:769] Sending connection context
0809 12:56:37.209018 (+ 221us) client_negotiation.cc:240] Negotiation successful
0809 12:56:37.209423 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":354,"threads_started":1}
I20260809 12:56:37.211982 26479 heartbeater.cc:347] Connected to a master server at 127.21.235.126:34287
I20260809 12:56:37.212282 26479 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:37.213009 26479 heartbeater.cc:511] Master 127.21.235.126:34287 requested a full tablet report, sending...
I20260809 12:56:37.214386 25965 ts_manager.cc:194] Registered new tserver with Master: 9e4af83874a24deaa29ae9fabb0027c7 (127.21.235.67:36315)
I20260809 12:56:37.215250 25965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:47611
I20260809 12:56:37.218254 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:56:37 dist-test-slave-gp6n krb5kdc[25916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280197, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 12:56:37 dist-test-slave-gp6n krb5kdc[25916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280197, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:37.272851 26179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:37.258479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44126 (local address 127.21.235.126:34287)
0809 12:56:37.258715 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:37.258724 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:37.259118 (+ 394us) server_negotiation.cc:407] Connection header received
0809 12:56:37.259335 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:37.259344 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:37.259416 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:37.259522 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:37.260898 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:37.261659 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:37.262985 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:37.263417 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:37.263559 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:37.267353 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:37.267358 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:37.267361 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:37.267397 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:37.270066 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:37.270962 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:37.270967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:37.270969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:37.271120 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:37.271522 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:37.271525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:37.271527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:37.271846 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:37.272034 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:37.272310 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 12:56:37.272508 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 12:56:37.273453 25965 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44126
I20260809 12:56:37.275445 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 26034
Aug 09 12:56:37 dist-test-slave-gp6n krb5kdc[25916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280197, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 12:56:37.291060 26493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:37.276457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42798 (local address 127.21.235.65:35157)
0809 12:56:37.277151 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:37.277158 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:37.277181 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:56:37.277244 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:37.277250 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:37.277453 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:37.277626 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:37.278922 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:37.279930 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:37.281149 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:37.281811 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:37.281957 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:37.285845 (+ 3888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:37.285855 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:37.285869 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:37.285901 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:37.288696 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:37.289291 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:37.289298 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:37.289300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:37.289425 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:37.289756 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:37.289758 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:37.289760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:37.290077 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:37.290299 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:37.290580 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 12:56:37.290769 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":288,"threads_started":1}
W20260809 12:56:37.292315 26171 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42798
W20260809 12:56:37.293449 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 26034: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:56:37.293671 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 26034
I20260809 12:56:37.315737 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 26185
Aug 09 12:56:37 dist-test-slave-gp6n krb5kdc[25916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280197, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.235.66@KRBTEST.COM
I20260809 12:56:37.332417 26494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:37.316607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47144 (local address 127.21.235.66:46123)
0809 12:56:37.317092 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:37.317099 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:37.317123 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:56:37.317184 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:37.317190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:37.317399 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:37.317542 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:37.318859 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:37.319910 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:37.321132 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:37.321681 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:37.321791 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:37.326393 (+ 4602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:37.326438 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:37.326463 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:37.326498 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:37.329709 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:37.330388 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:37.330395 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:37.330398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:37.330648 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:37.331068 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:37.331072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:37.331074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:37.331420 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:37.331670 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:37.331897 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 12:56:37.332124 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":235,"threads_started":1}
W20260809 12:56:37.333750 26323 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47144
W20260809 12:56:37.335073 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 26185: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:56:37.335202 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 26185
I20260809 12:56:37.353271 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 26334
Aug 09 12:56:37 dist-test-slave-gp6n krb5kdc[25916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280197, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.235.67@KRBTEST.COM
I20260809 12:56:37.369187 26495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:37.354157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59438 (local address 127.21.235.67:36315)
0809 12:56:37.354836 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:37.354843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:37.354873 (+ 30us) server_negotiation.cc:407] Connection header received
0809 12:56:37.354935 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:37.354941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:37.355154 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:37.355280 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:37.356634 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:37.357713 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:37.359013 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:37.359570 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:37.359664 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:37.363913 (+ 4249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:37.363924 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:37.363938 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:37.363967 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:37.366752 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:37.367340 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:37.367347 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:37.367349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:37.367481 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:37.367786 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:37.367789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:37.367791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:37.368114 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:37.368350 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:37.368599 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 12:56:37.368796 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":364,"threads_started":1}
W20260809 12:56:37.370584 26472 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59438
W20260809 12:56:37.371744 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 26334: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:56:37.371917 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 26334
I20260809 12:56:37.391283 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 25931
I20260809 12:56:37.392464 25997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:37.497114 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 25931
2026-08-09T12:56:37Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3802 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:56:37 dist-test-slave-gp6n krb5kdc[26499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:56:37 dist-test-slave-gp6n krb5kdc[26499](info): set up 2 sockets
Aug 09 12:56:37 dist-test-slave-gp6n krb5kdc[26499](info): commencing operation
krb5kdc: starting...
W20260809 12:56:38.648398 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:56:38 dist-test-slave-gp6n krb5kdc[26499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:56:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:56:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 12:56:38.862721 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:44159
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:44159
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:39.198516 26515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:39.198843 26515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:39.199008 26515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:39.206101 26515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:56:39.206241 26515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:39.206363 26515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:39.206485 26515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:56:39.206563 26515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:56:39.216387 26515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:44159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:44159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:39.217691 26515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:39.219020 26515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:39.231580 26520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:39.231624 26521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:39.231760 26524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:39.232604 26515 server_base.cc:1034] running on GCE node
I20260809 12:56:39.233457 26515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:39.235039 26515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:39.236359 26515 hybrid_clock.cc:648] HybridClock initialized: now 1786280199236297 us; error 43 us; skew 500 ppm
I20260809 12:56:39.236771 26515 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:39 dist-test-slave-gp6n krb5kdc[26499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280199, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:39.244848 26515 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:56:39.246373 26515 webserver.cc:460] Webserver started at http://127.21.235.126:35993/ using document root <none> and password file <none>
I20260809 12:56:39.247169 26515 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:39.247339 26515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:39.247680 26515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:39.250985 26515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "338ddf9d7ce5408987b421ead4347751"
format_stamp: "Formatted at 2026-08-09 12:56:39 on dist-test-slave-gp6n"
server_key: "c49933c3b5be964a1b427614610ed177"
server_key_iv: "16e1e0eabf2ddcebbc3a0db04034789e"
server_key_version: "encryptionkey@0"
I20260809 12:56:39.251935 26515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "338ddf9d7ce5408987b421ead4347751"
format_stamp: "Formatted at 2026-08-09 12:56:39 on dist-test-slave-gp6n"
server_key: "c49933c3b5be964a1b427614610ed177"
server_key_iv: "16e1e0eabf2ddcebbc3a0db04034789e"
server_key_version: "encryptionkey@0"
I20260809 12:56:39.257980 26515 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260809 12:56:39.261678 26532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:39.263131 26515 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:56:39.263336 26515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "338ddf9d7ce5408987b421ead4347751"
format_stamp: "Formatted at 2026-08-09 12:56:39 on dist-test-slave-gp6n"
server_key: "c49933c3b5be964a1b427614610ed177"
server_key_iv: "16e1e0eabf2ddcebbc3a0db04034789e"
server_key_version: "encryptionkey@0"
I20260809 12:56:39.263556 26515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:39.275823 26515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:39.280179 26515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:39.280475 26515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:39.298616 26515 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:44159
I20260809 12:56:39.298624 26594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:44159 every 8 connection(s)
I20260809 12:56:39.299969 26515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:56:39.302872 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 26515
I20260809 12:56:39.303113 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:56:39.303669 26595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:39.303848 22445 external_mini_cluster.cc:1428] Setting key eeb319e99f94bc6031685c3e4b24fb5d
I20260809 12:56:39.317875 26595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751: Bootstrap starting.
Aug 09 12:56:39 dist-test-slave-gp6n krb5kdc[26499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:39.321452 26595 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:39.322875 26595 log.cc:824] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:39.326064 26595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751: No bootstrap required, opened a new log
I20260809 12:56:39.328680 26600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:39.306005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59298 (local address 127.21.235.126:44159)
0809 12:56:39.306971 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:39.306987 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:39.307033 (+ 46us) server_negotiation.cc:407] Connection header received
0809 12:56:39.308713 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:39.308740 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:39.309232 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:39.309666 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:39.311265 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:39.313183 (+ 1918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:39.314739 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:39.315831 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:39.316028 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:39.320463 (+ 4435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:39.320474 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:39.320493 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:39.320547 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:39.324375 (+ 3828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:39.325032 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:39.325040 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:39.325047 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:39.325270 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:39.325625 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:39.325630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:39.325633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:39.326290 (+ 657us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:39.326648 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:39.326977 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 12:56:39.327384 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":460,"threads_started":1}
I20260809 12:56:39.335278 26595 raft_consensus.cc:348] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "338ddf9d7ce5408987b421ead4347751" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 44159 } }
I20260809 12:56:39.335618 26595 raft_consensus.cc:374] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:39.335690 26595 raft_consensus.cc:729] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 338ddf9d7ce5408987b421ead4347751, State: Initialized, Role: FOLLOWER
I20260809 12:56:39.336148 26595 consensus_queue.cc:260] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "338ddf9d7ce5408987b421ead4347751" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 44159 } }
I20260809 12:56:39.336355 26595 raft_consensus.cc:388] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:56:39.336452 26595 raft_consensus.cc:482] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:56:39.336570 26595 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:39.338935 26595 raft_consensus.cc:504] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "338ddf9d7ce5408987b421ead4347751" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 44159 } }
I20260809 12:56:39.339359 26595 leader_election.cc:302] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 338ddf9d7ce5408987b421ead4347751; no voters:
I20260809 12:56:39.339897 26595 leader_election.cc:288] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:56:39.340045 26602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:39.340764 26602 raft_consensus.cc:686] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751 [term 1 LEADER]: Becoming Leader. State: Replica: 338ddf9d7ce5408987b421ead4347751, State: Running, Role: LEADER
I20260809 12:56:39.341325 26602 consensus_queue.cc:237] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "338ddf9d7ce5408987b421ead4347751" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 44159 } }
I20260809 12:56:39.342005 26595 sys_catalog.cc:564] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:56:39.344439 26603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "338ddf9d7ce5408987b421ead4347751" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "338ddf9d7ce5408987b421ead4347751" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 44159 } } }
I20260809 12:56:39.344416 26604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 338ddf9d7ce5408987b421ead4347751. Latest consensus state: current_term: 1 leader_uuid: "338ddf9d7ce5408987b421ead4347751" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "338ddf9d7ce5408987b421ead4347751" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 44159 } } }
I20260809 12:56:39.344841 26603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751 [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:39.344846 26604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751 [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:39.346426 26608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:56:39.351732 26608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:56:39.360951 26608 catalog_manager.cc:1347] Generated new cluster ID: 0f1d165773b14897bc63b9e3786b75f9
I20260809 12:56:39.361091 26608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:56:39.378170 26608 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:56:39.379561 26608 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:56:39.390110 26608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 338ddf9d7ce5408987b421ead4347751: Generated new TSK 0
I20260809 12:56:39.391016 26608 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
I20260809 12:56:39.442842 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:44159
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:39.776336 26625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:39.776631 26625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:39.776728 26625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:39.784000 26625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:39.784116 26625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:39.784229 26625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:56:39.793972 26625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:44159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:39.795243 26625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:39.796519 26625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:39.809751 26632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:39.809787 26635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:39.809827 26631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:39.810606 26625 server_base.cc:1034] running on GCE node
I20260809 12:56:39.811251 26625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:39.812194 26625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:39.813400 26625 hybrid_clock.cc:648] HybridClock initialized: now 1786280199813366 us; error 30 us; skew 500 ppm
I20260809 12:56:39.813730 26625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:39 dist-test-slave-gp6n krb5kdc[26499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280199, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:39.817032 26625 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 12:56:39.818600 26625 webserver.cc:460] Webserver started at http://127.21.235.65:35203/ using document root <none> and password file <none>
I20260809 12:56:39.819267 26625 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:39.819396 26625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:39.819718 26625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:39.822584 26625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "2129ec4f408747e690ca0672fbb5ffdc"
format_stamp: "Formatted at 2026-08-09 12:56:39 on dist-test-slave-gp6n"
server_key: "3d7885db6408ee7f57426cd12fdaaf10"
server_key_iv: "aaa128528e03425549d1fdc40e607229"
server_key_version: "encryptionkey@0"
I20260809 12:56:39.823436 26625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "2129ec4f408747e690ca0672fbb5ffdc"
format_stamp: "Formatted at 2026-08-09 12:56:39 on dist-test-slave-gp6n"
server_key: "3d7885db6408ee7f57426cd12fdaaf10"
server_key_iv: "aaa128528e03425549d1fdc40e607229"
server_key_version: "encryptionkey@0"
I20260809 12:56:39.828719 26625 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:56:39.832131 26645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:39.833458 26625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:56:39.833626 26625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "2129ec4f408747e690ca0672fbb5ffdc"
format_stamp: "Formatted at 2026-08-09 12:56:39 on dist-test-slave-gp6n"
server_key: "3d7885db6408ee7f57426cd12fdaaf10"
server_key_iv: "aaa128528e03425549d1fdc40e607229"
server_key_version: "encryptionkey@0"
I20260809 12:56:39.833822 26625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:39.852293 26625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:39.856470 26625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:39.856750 26625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:39.857673 26625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:39.859057 26625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:39.859130 26625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:39.859297 26625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:39.859359 26625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:39.894639 26625 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:44161
I20260809 12:56:39.894726 26769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:44161 every 8 connection(s)
I20260809 12:56:39.895972 26625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:56:39.904527 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 26625
I20260809 12:56:39.904778 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:56:39.905526 22445 external_mini_cluster.cc:1428] Setting key 1752aff14e22c4557d6846fb05f0853a
Aug 09 12:56:39 dist-test-slave-gp6n krb5kdc[26499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280199, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:39.917883 26772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:39.898609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:43633 (local address 127.21.235.126:44159)
0809 12:56:39.899070 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:39.899075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:39.900071 (+ 996us) server_negotiation.cc:407] Connection header received
0809 12:56:39.901994 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:39.902001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:39.902094 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:39.902222 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:39.903832 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:39.904983 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:39.906802 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:39.907489 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:39.907626 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:39.911848 (+ 4222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:39.911854 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:39.911857 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:39.911888 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:39.914510 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:39.915219 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:39.915225 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:39.915229 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:39.915361 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:39.915961 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:39.915965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:39.915967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:39.916295 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:39.916450 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:39.917124 (+ 674us) server_negotiation.cc:299] Negotiation successful
0809 12:56:39.917381 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":253,"threads_started":1}
I20260809 12:56:39.918574 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:39.899108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:44159 (local address 127.21.235.65:43633)
0809 12:56:39.899885 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:39.899930 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:39.901713 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:39.902523 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:39.902532 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:39.903063 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:39.903648 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:39.903666 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:39.905151 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:39.905155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:39.906649 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:39.906663 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:39.907778 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:39.907787 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:39.908068 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:39.908783 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:39.908817 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:39.911603 (+ 2786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:39.914646 (+ 3043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:39.914656 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:39.914674 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:39.915094 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:39.915556 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:39.915562 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:39.915568 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:39.915841 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:39.916422 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:39.916428 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:39.916746 (+ 318us) client_negotiation.cc:769] Sending connection context
0809 12:56:39.916977 (+ 231us) client_negotiation.cc:240] Negotiation successful
0809 12:56:39.917362 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":313,"threads_started":1}
I20260809 12:56:39.920143 26770 heartbeater.cc:347] Connected to a master server at 127.21.235.126:44159
I20260809 12:56:39.920461 26770 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:39.921180 26770 heartbeater.cc:511] Master 127.21.235.126:44159 requested a full tablet report, sending...
I20260809 12:56:39.923271 26553 ts_manager.cc:194] Registered new tserver with Master: 2129ec4f408747e690ca0672fbb5ffdc (127.21.235.65:44161)
I20260809 12:56:39.924774 26553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:43633
WARNING: no policy specified for kudu/127.21.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
I20260809 12:56:39.955869 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:44159
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:40.282194 26776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:40.282645 26776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:40.282812 26776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:40.290068 26776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:40.290174 26776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:40.290364 26776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:56:40.300166 26776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:44159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:40.301376 26776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:40.302827 26776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:40.317970 26781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:40.317950 26782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:40.317974 26784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:40.319334 26776 server_base.cc:1034] running on GCE node
I20260809 12:56:40.319926 26776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:40.320843 26776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:40.322042 26776 hybrid_clock.cc:648] HybridClock initialized: now 1786280200321977 us; error 53 us; skew 500 ppm
I20260809 12:56:40.322355 26776 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:40 dist-test-slave-gp6n krb5kdc[26499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280200, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:40.325692 26776 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 12:56:40.327163 26776 webserver.cc:460] Webserver started at http://127.21.235.66:44535/ using document root <none> and password file <none>
I20260809 12:56:40.327850 26776 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:40.328001 26776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:40.328315 26776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:40.331441 26776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "01ba8c9f78ae4233935a019141da3ac8"
format_stamp: "Formatted at 2026-08-09 12:56:40 on dist-test-slave-gp6n"
server_key: "feba4bec810a0774b97f83fea4dfe50f"
server_key_iv: "915e364769d9fec5cdb4bd63893361c8"
server_key_version: "encryptionkey@0"
I20260809 12:56:40.332316 26776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "01ba8c9f78ae4233935a019141da3ac8"
format_stamp: "Formatted at 2026-08-09 12:56:40 on dist-test-slave-gp6n"
server_key: "feba4bec810a0774b97f83fea4dfe50f"
server_key_iv: "915e364769d9fec5cdb4bd63893361c8"
server_key_version: "encryptionkey@0"
I20260809 12:56:40.337572 26776 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:56:40.341153 26791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:40.342527 26776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 12:56:40.342689 26776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "01ba8c9f78ae4233935a019141da3ac8"
format_stamp: "Formatted at 2026-08-09 12:56:40 on dist-test-slave-gp6n"
server_key: "feba4bec810a0774b97f83fea4dfe50f"
server_key_iv: "915e364769d9fec5cdb4bd63893361c8"
server_key_version: "encryptionkey@0"
I20260809 12:56:40.342886 26776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:40.357304 26776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:40.361629 26776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:40.361931 26776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:40.362898 26776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:40.364168 26776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:40.364272 26776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:40.364423 26776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:40.364500 26776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:40.396317 26776 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:34389
I20260809 12:56:40.396335 26911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:34389 every 8 connection(s)
I20260809 12:56:40.397706 26776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:56:40.404552 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 26776
I20260809 12:56:40.404758 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:56:40.405181 22445 external_mini_cluster.cc:1428] Setting key d49061c6ab202d5e9355a9d48ef5cf25
Aug 09 12:56:40 dist-test-slave-gp6n krb5kdc[26499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280200, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:40.419072 26772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:40.400308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:59519 (local address 127.21.235.126:44159)
0809 12:56:40.400550 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:40.400557 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:40.401745 (+ 1188us) server_negotiation.cc:407] Connection header received
0809 12:56:40.403644 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:40.403648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:40.403718 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:40.403800 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:40.405559 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:40.406454 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:40.408175 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:40.408975 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:40.409126 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:40.412751 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:40.412760 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:40.412767 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:40.412814 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:40.415866 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:40.416747 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:40.416758 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:40.416761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:40.416887 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:40.417327 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:40.417333 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:40.417336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:40.417599 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:40.417754 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:40.418464 (+ 710us) server_negotiation.cc:299] Negotiation successful
0809 12:56:40.418709 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:56:40.420133 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:40.400857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:44159 (local address 127.21.235.66:59519)
0809 12:56:40.401585 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:40.401624 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:40.403378 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:40.404009 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:40.404017 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:40.404657 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:40.405380 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:40.405400 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:40.406606 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:40.406610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:40.408026 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:40.408039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:40.409210 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:40.409216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:40.409446 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:40.410183 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:40.410207 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:40.412524 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:40.416019 (+ 3495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:40.416028 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:40.416044 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:40.416622 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:40.417008 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:40.417013 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:40.417016 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:40.417212 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:40.417730 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:40.417737 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:40.418081 (+ 344us) client_negotiation.cc:769] Sending connection context
0809 12:56:40.418344 (+ 263us) client_negotiation.cc:240] Negotiation successful
0809 12:56:40.418904 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":277,"threads_started":1}
I20260809 12:56:40.421643 26912 heartbeater.cc:347] Connected to a master server at 127.21.235.126:44159
I20260809 12:56:40.421963 26912 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:40.422730 26912 heartbeater.cc:511] Master 127.21.235.126:44159 requested a full tablet report, sending...
I20260809 12:56:40.424201 26553 ts_manager.cc:194] Registered new tserver with Master: 01ba8c9f78ae4233935a019141da3ac8 (127.21.235.66:34389)
I20260809 12:56:40.425029 26553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:59519
WARNING: no policy specified for kudu/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
I20260809 12:56:40.457758 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:44159
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:40.786530 26918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:40.786813 26918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:40.786911 26918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:40.794111 26918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:40.794232 26918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:40.794349 26918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:56:40.804345 26918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:44159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:40.805791 26918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:40.807171 26918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:40.820420 26926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:40.820461 26924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:40.820680 26923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:40.821462 26918 server_base.cc:1034] running on GCE node
I20260809 12:56:40.822072 26918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:40.823115 26918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:40.824311 26918 hybrid_clock.cc:648] HybridClock initialized: now 1786280200824246 us; error 54 us; skew 500 ppm
I20260809 12:56:40.824641 26918 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:40 dist-test-slave-gp6n krb5kdc[26499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280200, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:40.827993 26918 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 12:56:40.829545 26918 webserver.cc:460] Webserver started at http://127.21.235.67:43913/ using document root <none> and password file <none>
I20260809 12:56:40.830193 26918 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:40.830294 26918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:40.830611 26918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:40.833485 26918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "0aa7cf05113c41498d36d703460fc043"
format_stamp: "Formatted at 2026-08-09 12:56:40 on dist-test-slave-gp6n"
server_key: "68f45cef704ddde993598daec213fca5"
server_key_iv: "f916bd5a02f29f133172a94a9c88f257"
server_key_version: "encryptionkey@0"
I20260809 12:56:40.834208 26918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "0aa7cf05113c41498d36d703460fc043"
format_stamp: "Formatted at 2026-08-09 12:56:40 on dist-test-slave-gp6n"
server_key: "68f45cef704ddde993598daec213fca5"
server_key_iv: "f916bd5a02f29f133172a94a9c88f257"
server_key_version: "encryptionkey@0"
I20260809 12:56:40.839471 26918 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:56:40.843041 26934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:40.844450 26918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 12:56:40.844615 26918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "0aa7cf05113c41498d36d703460fc043"
format_stamp: "Formatted at 2026-08-09 12:56:40 on dist-test-slave-gp6n"
server_key: "68f45cef704ddde993598daec213fca5"
server_key_iv: "f916bd5a02f29f133172a94a9c88f257"
server_key_version: "encryptionkey@0"
I20260809 12:56:40.844830 26918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:40.863389 26918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:40.868073 26918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:40.868413 26918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:40.869468 26918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:40.870929 26918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:40.871002 26918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:40.871156 26918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:40.871213 26918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:40.906293 26918 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:38133
I20260809 12:56:40.906342 27063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:38133 every 8 connection(s)
I20260809 12:56:40.907645 26918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:56:40.908764 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 26918
I20260809 12:56:40.908995 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:56:40.909543 22445 external_mini_cluster.cc:1428] Setting key 42de76c55a67f7c3b973a784e839d68f
Aug 09 12:56:40 dist-test-slave-gp6n krb5kdc[26499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280200, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:40.927724 26770 heartbeater.cc:503] Master 127.21.235.126:44159 was elected leader, sending a full tablet report...
I20260809 12:56:40.930486 26772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:40.910660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:54395 (local address 127.21.235.126:44159)
0809 12:56:40.910838 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:40.910844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:40.912248 (+ 1404us) server_negotiation.cc:407] Connection header received
0809 12:56:40.914450 (+ 2202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:40.914456 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:40.914537 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:40.914618 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:40.916116 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:40.916896 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:40.919086 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:40.919524 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:40.919626 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:40.923189 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:40.923194 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:40.923198 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:40.923233 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:40.926733 (+ 3500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:40.927676 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:40.927682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:40.927686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:40.927882 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:40.928502 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:40.928507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:40.928510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:40.928869 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:40.929021 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:40.929967 (+ 946us) server_negotiation.cc:299] Negotiation successful
0809 12:56:40.930166 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 12:56:40.931514 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:40.911206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:44159 (local address 127.21.235.67:54395)
0809 12:56:40.912065 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:40.912103 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:40.914169 (+ 2066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:40.914827 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:40.914836 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:40.915360 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:40.915931 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:40.915960 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:40.917041 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:40.917045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:40.918932 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:40.918960 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:40.919720 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:40.919724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:40.919920 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:40.920595 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:40.920624 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:40.922965 (+ 2341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:40.926916 (+ 3951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:40.926924 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:40.926939 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:40.927508 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:40.928027 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:40.928031 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:40.928033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:40.928345 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:40.929043 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:40.929051 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:40.929580 (+ 529us) client_negotiation.cc:769] Sending connection context
0809 12:56:40.929847 (+ 267us) client_negotiation.cc:240] Negotiation successful
0809 12:56:40.930274 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":304,"threads_started":1}
I20260809 12:56:40.933017 27064 heartbeater.cc:347] Connected to a master server at 127.21.235.126:44159
I20260809 12:56:40.933372 27064 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:40.934140 27064 heartbeater.cc:511] Master 127.21.235.126:44159 requested a full tablet report, sending...
I20260809 12:56:40.935669 26553 ts_manager.cc:194] Registered new tserver with Master: 0aa7cf05113c41498d36d703460fc043 (127.21.235.67:38133)
I20260809 12:56:40.936512 26553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:54395
I20260809 12:56:40.939199 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:56:40 dist-test-slave-gp6n krb5kdc[26499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 12:56:40 dist-test-slave-gp6n krb5kdc[26499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 12:56:40.995128 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:40.978642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43528 (local address 127.21.235.65:44161)
0809 12:56:40.979308 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:40.979315 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:40.979344 (+ 29us) server_negotiation.cc:407] Connection header received
0809 12:56:40.979448 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:40.979454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:40.979670 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:40.979801 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:40.981242 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:40.982307 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:40.983676 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:40.984285 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:40.984421 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:40.988939 (+ 4518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:40.988953 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:40.988969 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:40.989008 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:40.992243 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:40.993027 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:40.993035 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:40.993037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:40.993184 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:40.993564 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:40.993567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:40.993570 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:40.993972 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:40.994223 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:40.994579 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 12:56:40.994806 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":374,"threads_started":1}
Aug 09 12:56:41 dist-test-slave-gp6n krb5kdc[26499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:41.013499 26772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:40.999770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59310 (local address 127.21.235.126:44159)
0809 12:56:41.000030 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:41.000039 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:41.000184 (+ 145us) server_negotiation.cc:407] Connection header received
0809 12:56:41.000318 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:41.000322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:41.000384 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:41.000482 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:41.001712 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:41.002522 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:41.003802 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:41.004234 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:41.004333 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:41.008213 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:41.008219 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:41.008222 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:41.008256 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:41.010899 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:41.011496 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:41.011500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:41.011502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:41.011629 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:41.012042 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:41.012046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:41.012048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:41.012373 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:41.012642 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:41.012973 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 12:56:41.013197 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260809 12:56:41.014153 26553 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59310
I20260809 12:56:41.015223 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 26625
I20260809 12:56:41.027665 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:41.016203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43544 (local address 127.21.235.65:44161)
0809 12:56:41.016373 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:41.016377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:41.016626 (+ 249us) server_negotiation.cc:407] Connection header received
0809 12:56:41.016748 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:41.016752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:41.016811 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:41.016889 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:41.018123 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:41.018854 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:41.020166 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:41.020615 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:41.020722 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:41.022872 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:41.022877 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:41.022881 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:41.022908 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:41.025518 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:41.026118 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:41.026122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:41.026124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:41.026243 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:41.026613 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:41.026616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:41.026618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:41.026851 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:41.027032 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:41.027279 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 12:56:41.027434 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 12:56:41.028314 26764 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:41.131367 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 26625
I20260809 12:56:41.152367 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 26776
Aug 09 12:56:41 dist-test-slave-gp6n krb5kdc[26499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.66@KRBTEST.COM
I20260809 12:56:41.168812 27079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:41.153397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46702 (local address 127.21.235.66:34389)
0809 12:56:41.154008 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:41.154016 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:41.154043 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:56:41.154107 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:41.154113 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:41.154339 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:41.154527 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:41.155914 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:41.157001 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:41.158234 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:41.158904 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:41.159005 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:41.163305 (+ 4300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:41.163324 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:41.163338 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:41.163372 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:41.166226 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:41.166881 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:41.166888 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:41.166890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:41.167037 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:41.167379 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:41.167382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:41.167384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:41.167765 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:41.168019 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:41.168218 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 12:56:41.168503 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":303,"threads_started":1}
I20260809 12:56:41.170084 26907 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:41.255079 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 26776
I20260809 12:56:41.274622 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 26918
Aug 09 12:56:41 dist-test-slave-gp6n krb5kdc[26499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.67@KRBTEST.COM
I20260809 12:56:41.291931 27081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:41.275869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50630 (local address 127.21.235.67:38133)
0809 12:56:41.276539 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:41.276546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:41.276576 (+ 30us) server_negotiation.cc:407] Connection header received
0809 12:56:41.276640 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:41.276647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:41.276858 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:41.277053 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:41.278378 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:41.279546 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:41.280863 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:41.281469 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:41.281614 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:41.285936 (+ 4322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:41.285946 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:41.285963 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:41.285998 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:41.288888 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:41.289587 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:41.289594 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:41.289597 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:41.289734 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:41.290112 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:41.290118 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:41.290121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:41.290571 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:41.290892 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:41.291284 (+ 392us) server_negotiation.cc:299] Negotiation successful
0809 12:56:41.291577 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":341,"threads_started":1}
I20260809 12:56:41.293239 27059 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:41.381305 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 26918
I20260809 12:56:41.400317 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 26515
I20260809 12:56:41.401638 26589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:41.506706 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 26515
2026-08-09T12:56:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4011 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:56:41 dist-test-slave-gp6n krb5kdc[27086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:56:41 dist-test-slave-gp6n krb5kdc[27086](info): set up 2 sockets
Aug 09 12:56:41 dist-test-slave-gp6n krb5kdc[27086](info): commencing operation
krb5kdc: starting...
W20260809 12:56:42.672520 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:56:42 dist-test-slave-gp6n krb5kdc[27086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:56:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:56:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 12:56:42.888188 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:34993
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:42749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:34993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:43.227404 27100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:43.227777 27100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:43.227964 27100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:43.235983 27100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:56:43.236136 27100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:43.236243 27100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:43.236351 27100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:56:43.236447 27100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:56:43.246475 27100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:42749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:34993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:34993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:43.247782 27100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:43.249131 27100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:43.262008 27111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:43.261960 27109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:43.262034 27108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:43.262723 27100 server_base.cc:1034] running on GCE node
I20260809 12:56:43.263428 27100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:43.264673 27100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:43.265888 27100 hybrid_clock.cc:648] HybridClock initialized: now 1786280203265842 us; error 44 us; skew 500 ppm
I20260809 12:56:43.266263 27100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:43 dist-test-slave-gp6n krb5kdc[27086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280203, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:43.269500 27100 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:56:43.270864 27100 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:43.270999 27100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:43.271343 27100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:43.274492 27100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "6ad48c7c97584c379f68551f8f6668a0"
format_stamp: "Formatted at 2026-08-09 12:56:43 on dist-test-slave-gp6n"
server_key: "db174783ff71b005c62e3886b07ce96b"
server_key_iv: "6e883cb4e7a852f17ad11672042ae748"
server_key_version: "encryptionkey@0"
I20260809 12:56:43.275339 27100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "6ad48c7c97584c379f68551f8f6668a0"
format_stamp: "Formatted at 2026-08-09 12:56:43 on dist-test-slave-gp6n"
server_key: "db174783ff71b005c62e3886b07ce96b"
server_key_iv: "6e883cb4e7a852f17ad11672042ae748"
server_key_version: "encryptionkey@0"
I20260809 12:56:43.280645 27100 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:56:43.284242 27119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:43.285629 27100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:56:43.285916 27100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "6ad48c7c97584c379f68551f8f6668a0"
format_stamp: "Formatted at 2026-08-09 12:56:43 on dist-test-slave-gp6n"
server_key: "db174783ff71b005c62e3886b07ce96b"
server_key_iv: "6e883cb4e7a852f17ad11672042ae748"
server_key_version: "encryptionkey@0"
I20260809 12:56:43.286176 27100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:43.300822 27100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:43.304813 27100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:43.305089 27100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:43.323968 27100 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:34993
I20260809 12:56:43.323993 27174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:34993 every 8 connection(s)
I20260809 12:56:43.324896 27100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:56:43.328389 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 27100
I20260809 12:56:43.328646 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:56:43.328857 27175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:43.329478 22445 external_mini_cluster.cc:1428] Setting key f13d6da9d55b9a2fec0412ac9a56c341
I20260809 12:56:43.343523 27175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0: Bootstrap starting.
Aug 09 12:56:43 dist-test-slave-gp6n krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:43.347530 27175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:43.348613 27175 log.cc:824] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:43.351572 27175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0: No bootstrap required, opened a new log
I20260809 12:56:43.355221 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:43.331707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54748 (local address 127.21.235.126:34993)
0809 12:56:43.332687 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:43.332700 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:43.332742 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:56:43.335438 (+ 2696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:43.335462 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:43.335937 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:43.336314 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:43.338308 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:43.339892 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:43.341747 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:43.342584 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:43.342739 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:43.347697 (+ 4958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:43.347707 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:43.347722 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:43.347763 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:43.350986 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:43.351693 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:43.351700 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:43.351706 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:43.351865 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:43.352270 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:43.352281 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:43.352289 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:43.352912 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:43.353259 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:43.353703 (+ 444us) server_negotiation.cc:299] Negotiation successful
0809 12:56:43.354151 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":459,"threads_started":1}
I20260809 12:56:43.360229 27175 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ad48c7c97584c379f68551f8f6668a0" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34993 } }
I20260809 12:56:43.360625 27175 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:43.360730 27175 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad48c7c97584c379f68551f8f6668a0, State: Initialized, Role: FOLLOWER
I20260809 12:56:43.361352 27175 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ad48c7c97584c379f68551f8f6668a0" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34993 } }
I20260809 12:56:43.361594 27175 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:56:43.361742 27175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:56:43.361922 27175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:43.364130 27175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ad48c7c97584c379f68551f8f6668a0" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34993 } }
I20260809 12:56:43.364617 27175 leader_election.cc:302] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ad48c7c97584c379f68551f8f6668a0; no voters:
I20260809 12:56:43.365160 27175 leader_election.cc:288] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:56:43.365293 27180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:43.366082 27180 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0 [term 1 LEADER]: Becoming Leader. State: Replica: 6ad48c7c97584c379f68551f8f6668a0, State: Running, Role: LEADER
I20260809 12:56:43.366670 27180 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ad48c7c97584c379f68551f8f6668a0" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34993 } }
I20260809 12:56:43.367507 27175 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:56:43.369778 27181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ad48c7c97584c379f68551f8f6668a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ad48c7c97584c379f68551f8f6668a0" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34993 } } }
I20260809 12:56:43.370134 27181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0 [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:43.369838 27182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ad48c7c97584c379f68551f8f6668a0. Latest consensus state: current_term: 1 leader_uuid: "6ad48c7c97584c379f68551f8f6668a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ad48c7c97584c379f68551f8f6668a0" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34993 } } }
I20260809 12:56:43.370285 27182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0 [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:43.372025 27186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:56:43.378623 27186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:56:43.387341 27186 catalog_manager.cc:1347] Generated new cluster ID: 6bfaaec2b2b84bffad78515891684d7b
I20260809 12:56:43.387511 27186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:56:43.424121 27186 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:56:43.425536 27186 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:56:43.431566 27186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0: Generated new TSK 0
I20260809 12:56:43.432375 27186 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
I20260809 12:56:43.491400 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:34993
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:42749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:43.823187 27206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:43.823464 27206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:43.823632 27206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:43.830825 27206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:43.830948 27206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:43.831141 27206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:56:43.840749 27206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:42749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:34993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:43.842073 27206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:43.843423 27206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:43.857015 27212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:43.857302 27214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:43.857016 27211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:43.859766 27206 server_base.cc:1034] running on GCE node
I20260809 12:56:43.860523 27206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:43.861543 27206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:43.862776 27206 hybrid_clock.cc:648] HybridClock initialized: now 1786280203862738 us; error 61 us; skew 500 ppm
I20260809 12:56:43.863130 27206 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:43 dist-test-slave-gp6n krb5kdc[27086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280203, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:43.866328 27206 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 12:56:43.867583 27206 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:43.867720 27206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:43.868117 27206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:43.871217 27206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "06f0fe96a80f4f86ac8a64f477e35c19"
format_stamp: "Formatted at 2026-08-09 12:56:43 on dist-test-slave-gp6n"
server_key: "48dd251ffefd394c28977f2cf455f946"
server_key_iv: "f6dabcfd7e5f717582959dd6841df787"
server_key_version: "encryptionkey@0"
I20260809 12:56:43.872141 27206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "06f0fe96a80f4f86ac8a64f477e35c19"
format_stamp: "Formatted at 2026-08-09 12:56:43 on dist-test-slave-gp6n"
server_key: "48dd251ffefd394c28977f2cf455f946"
server_key_iv: "f6dabcfd7e5f717582959dd6841df787"
server_key_version: "encryptionkey@0"
I20260809 12:56:43.877146 27206 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 12:56:43.880517 27220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:43.881800 27206 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:56:43.881959 27206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "06f0fe96a80f4f86ac8a64f477e35c19"
format_stamp: "Formatted at 2026-08-09 12:56:43 on dist-test-slave-gp6n"
server_key: "48dd251ffefd394c28977f2cf455f946"
server_key_iv: "f6dabcfd7e5f717582959dd6841df787"
server_key_version: "encryptionkey@0"
I20260809 12:56:43.882174 27206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:43.907768 27206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:43.911746 27206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:43.912043 27206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:43.913004 27206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:43.914345 27206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:43.914494 27206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:43.914584 27206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:43.914642 27206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:43.948539 27206 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:36219
I20260809 12:56:43.948640 27346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:36219 every 8 connection(s)
I20260809 12:56:43.949447 27206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:56:43.954285 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 27206
I20260809 12:56:43.954555 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:56:43.955196 22445 external_mini_cluster.cc:1428] Setting key 62f70f35d4d7136602bd5506de7fd36c
Aug 09 12:56:43 dist-test-slave-gp6n krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280203, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:43.971362 27349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:43.951912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:56971 (local address 127.21.235.126:34993)
0809 12:56:43.952356 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:43.952360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:43.953378 (+ 1018us) server_negotiation.cc:407] Connection header received
0809 12:56:43.955367 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:43.955373 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:43.955467 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:43.955588 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:43.957338 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:43.958060 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:43.959729 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:43.960129 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:43.960267 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:43.964297 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:43.964305 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:43.964313 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:43.964360 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:43.967945 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:43.968724 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:43.968732 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:43.968738 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:43.968894 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:43.969334 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:43.969342 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:43.969348 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:43.969728 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:43.969882 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:43.970586 (+ 704us) server_negotiation.cc:299] Negotiation successful
0809 12:56:43.970844 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":228,"threads_started":1}
I20260809 12:56:43.971949 27350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:43.952514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:34993 (local address 127.21.235.65:56971)
0809 12:56:43.953210 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:43.953258 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:43.955093 (+ 1835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:43.955828 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:43.955835 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:43.956469 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:43.957175 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:43.957195 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:43.958239 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:43.958242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:43.959562 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:43.959575 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:43.960352 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:43.960360 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:43.960607 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:43.961294 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:43.961316 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:43.964038 (+ 2722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:43.968084 (+ 4046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:43.968090 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:43.968103 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:43.968568 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:43.969021 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:43.969024 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:43.969026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:43.969210 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:43.969848 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:43.969854 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:43.970161 (+ 307us) client_negotiation.cc:769] Sending connection context
0809 12:56:43.970379 (+ 218us) client_negotiation.cc:240] Negotiation successful
0809 12:56:43.970841 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":290,"threads_started":1}
I20260809 12:56:43.973753 27347 heartbeater.cc:347] Connected to a master server at 127.21.235.126:34993
I20260809 12:56:43.974073 27347 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:43.974853 27347 heartbeater.cc:511] Master 127.21.235.126:34993 requested a full tablet report, sending...
I20260809 12:56:43.976961 27136 ts_manager.cc:194] Registered new tserver with Master: 06f0fe96a80f4f86ac8a64f477e35c19 (127.21.235.65:36219)
WARNING: no policy specified for kudu/127.21.235.66@KRBTEST.COM; defaulting to no policy
I20260809 12:56:43.978570 27136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:56971
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
I20260809 12:56:44.009409 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:34993
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:42749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:44.346490 27354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:44.346796 27354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:44.346983 27354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:44.354247 27354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:44.354355 27354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:44.354528 27354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:56:44.364221 27354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:42749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:34993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:44.365425 27354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:44.366748 27354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:44.380616 27360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:44.380663 27363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:44.380693 27361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:44.381484 27354 server_base.cc:1034] running on GCE node
I20260809 12:56:44.382030 27354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:44.382978 27354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:44.384260 27354 hybrid_clock.cc:648] HybridClock initialized: now 1786280204384207 us; error 43 us; skew 500 ppm
I20260809 12:56:44.384649 27354 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:44 dist-test-slave-gp6n krb5kdc[27086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280204, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:44.391736 27354 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 12:56:44.392966 27354 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:44.393105 27354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:44.393424 27354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:44.396493 27354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "f7257b096c6145e3abdf36db21ca16e3"
format_stamp: "Formatted at 2026-08-09 12:56:44 on dist-test-slave-gp6n"
server_key: "c2d77f601f7225ca3bdef7886eb5cf4e"
server_key_iv: "b8d945d143b6e53b2b95c178d6f808d8"
server_key_version: "encryptionkey@0"
I20260809 12:56:44.397302 27354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "f7257b096c6145e3abdf36db21ca16e3"
format_stamp: "Formatted at 2026-08-09 12:56:44 on dist-test-slave-gp6n"
server_key: "c2d77f601f7225ca3bdef7886eb5cf4e"
server_key_iv: "b8d945d143b6e53b2b95c178d6f808d8"
server_key_version: "encryptionkey@0"
I20260809 12:56:44.402612 27354 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:56:44.406032 27369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:44.407389 27354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:56:44.407577 27354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "f7257b096c6145e3abdf36db21ca16e3"
format_stamp: "Formatted at 2026-08-09 12:56:44 on dist-test-slave-gp6n"
server_key: "c2d77f601f7225ca3bdef7886eb5cf4e"
server_key_iv: "b8d945d143b6e53b2b95c178d6f808d8"
server_key_version: "encryptionkey@0"
I20260809 12:56:44.407804 27354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:44.448410 27354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:44.452894 27354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:44.453197 27354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:44.454227 27354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:44.455639 27354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:44.455762 27354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:44.455893 27354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:44.455981 27354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:44.490552 27354 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:39989
I20260809 12:56:44.490636 27491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:39989 every 8 connection(s)
I20260809 12:56:44.491526 27354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:56:44.501276 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 27354
I20260809 12:56:44.501502 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:56:44.502090 22445 external_mini_cluster.cc:1428] Setting key e8fd554a35580fe011f4dda2449fe564
Aug 09 12:56:44 dist-test-slave-gp6n krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280204, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:44.513770 27494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:44.494246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:43503 (local address 127.21.235.126:34993)
0809 12:56:44.494771 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:44.494777 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:44.495820 (+ 1043us) server_negotiation.cc:407] Connection header received
0809 12:56:44.497629 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:44.497634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:44.497704 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:44.497806 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:44.499468 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:44.500174 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:44.501875 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:44.502551 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:44.502684 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:44.506443 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:44.506452 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:44.506460 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:44.506515 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:44.510184 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:44.510959 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:44.510967 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:44.510973 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:44.511158 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:44.511605 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:44.511612 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:44.511618 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:44.512004 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:44.512158 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:44.513051 (+ 893us) server_negotiation.cc:299] Negotiation successful
0809 12:56:44.513324 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":279,"threads_started":1}
I20260809 12:56:44.514492 27495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:44.494890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:34993 (local address 127.21.235.66:43503)
0809 12:56:44.495643 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:44.495682 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:44.497358 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:44.498028 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:44.498036 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:44.498707 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:44.499291 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:44.499316 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:44.500361 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:44.500365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:44.501733 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:44.501744 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:44.502797 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:44.502805 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:44.503111 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:44.503837 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:44.503889 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:44.506177 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:44.510328 (+ 4151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:44.510334 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:44.510347 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:44.510846 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:44.511303 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:44.511306 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:44.511309 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:44.511484 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:44.512222 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:44.512234 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:44.512606 (+ 372us) client_negotiation.cc:769] Sending connection context
0809 12:56:44.512883 (+ 277us) client_negotiation.cc:240] Negotiation successful
0809 12:56:44.513258 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":293,"threads_started":1}
I20260809 12:56:44.516002 27492 heartbeater.cc:347] Connected to a master server at 127.21.235.126:34993
I20260809 12:56:44.516310 27492 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:44.516963 27492 heartbeater.cc:511] Master 127.21.235.126:34993 requested a full tablet report, sending...
I20260809 12:56:44.518390 27136 ts_manager.cc:194] Registered new tserver with Master: f7257b096c6145e3abdf36db21ca16e3 (127.21.235.66:39989)
I20260809 12:56:44.519239 27136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:43503
WARNING: no policy specified for kudu/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
I20260809 12:56:44.554122 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:34993
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:42749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:44.882421 27499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:44.882735 27499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:44.882908 27499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:44.890273 27499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:44.890439 27499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:44.890612 27499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:56:44.900385 27499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:42749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:34993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:44.901686 27499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:44.903047 27499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:44.915962 27505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:44.916060 27508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:44.916002 27506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:44.916971 27499 server_base.cc:1034] running on GCE node
I20260809 12:56:44.917526 27499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:44.918391 27499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:44.919646 27499 hybrid_clock.cc:648] HybridClock initialized: now 1786280204919613 us; error 30 us; skew 500 ppm
I20260809 12:56:44.919976 27499 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:44 dist-test-slave-gp6n krb5kdc[27086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280204, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:44.923224 27499 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 12:56:44.924386 27499 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:44.924513 27499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:44.924865 27499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:44.927937 27499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "8348fa2c88f1462e92f633b861a3289e"
format_stamp: "Formatted at 2026-08-09 12:56:44 on dist-test-slave-gp6n"
server_key: "e9f32d6214293b8c8fa4b242c72c6bc1"
server_key_iv: "74735ff6643e8dd49647e808a6cdea7b"
server_key_version: "encryptionkey@0"
I20260809 12:56:44.928702 27499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "8348fa2c88f1462e92f633b861a3289e"
format_stamp: "Formatted at 2026-08-09 12:56:44 on dist-test-slave-gp6n"
server_key: "e9f32d6214293b8c8fa4b242c72c6bc1"
server_key_iv: "74735ff6643e8dd49647e808a6cdea7b"
server_key_version: "encryptionkey@0"
I20260809 12:56:44.933863 27499 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:56:44.937492 27516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:44.938936 27499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:56:44.939110 27499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "8348fa2c88f1462e92f633b861a3289e"
format_stamp: "Formatted at 2026-08-09 12:56:44 on dist-test-slave-gp6n"
server_key: "e9f32d6214293b8c8fa4b242c72c6bc1"
server_key_iv: "74735ff6643e8dd49647e808a6cdea7b"
server_key_version: "encryptionkey@0"
I20260809 12:56:44.939333 27499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:44.974524 27499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:44.978915 27499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:44.979193 27499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:44.980180 27499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:44.981537 27347 heartbeater.cc:503] Master 127.21.235.126:34993 was elected leader, sending a full tablet report...
I20260809 12:56:44.981629 27499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:44.981719 27499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:44.981844 27499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:44.981928 27499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:45.016698 27499 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:42289
I20260809 12:56:45.016789 27639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:42289 every 8 connection(s)
I20260809 12:56:45.017619 27499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:56:45.026659 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 27499
I20260809 12:56:45.026865 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:56:45.027283 22445 external_mini_cluster.cc:1428] Setting key c3d907483e0311a6a58e9868ed0641eb
Aug 09 12:56:45 dist-test-slave-gp6n krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280204, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:45.037866 27642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:45.020208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:39235 (local address 127.21.235.126:34993)
0809 12:56:45.020714 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:45.020720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:45.021752 (+ 1032us) server_negotiation.cc:407] Connection header received
0809 12:56:45.023579 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:45.023584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:45.023652 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:45.023749 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:45.025207 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.025963 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.027745 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.028199 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.028350 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:45.032207 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:45.032211 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:45.032215 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:45.032246 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:45.034808 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:45.035540 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:45.035544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:45.035547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:45.035689 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:45.036127 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:45.036131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:45.036134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:45.036389 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:45.036559 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:45.037256 (+ 697us) server_negotiation.cc:299] Negotiation successful
0809 12:56:45.037494 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":213,"threads_started":1}
I20260809 12:56:45.038815 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:45.020877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:34993 (local address 127.21.235.67:39235)
0809 12:56:45.021587 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:45.021633 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:45.023328 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:45.023953 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:45.023961 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:45.024478 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:45.025051 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:45.025068 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.026085 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.026088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:45.027601 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:45.027613 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.028417 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.028422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:45.028703 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:45.029474 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:45.029507 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:45.032001 (+ 2494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:45.034942 (+ 2941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:45.034951 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:45.034967 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:45.035393 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:45.035801 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:45.035806 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:45.035809 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:45.035998 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:45.036533 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:45.036541 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:45.036871 (+ 330us) client_negotiation.cc:769] Sending connection context
0809 12:56:45.037144 (+ 273us) client_negotiation.cc:240] Negotiation successful
0809 12:56:45.037652 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":279,"threads_started":1}
I20260809 12:56:45.040411 27640 heartbeater.cc:347] Connected to a master server at 127.21.235.126:34993
I20260809 12:56:45.040748 27640 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:45.041472 27640 heartbeater.cc:511] Master 127.21.235.126:34993 requested a full tablet report, sending...
I20260809 12:56:45.043025 27136 ts_manager.cc:194] Registered new tserver with Master: 8348fa2c88f1462e92f633b861a3289e (127.21.235.67:42289)
I20260809 12:56:45.043907 27136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:39235
I20260809 12:56:45.054966 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:56:45.072576 27642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:45.059555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54752 (local address 127.21.235.126:34993)
0809 12:56:45.059751 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:45.059756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:45.060294 (+ 538us) server_negotiation.cc:407] Connection header received
0809 12:56:45.060509 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:45.060513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:45.060575 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:45.060674 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:45.062061 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.062882 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.064202 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.064615 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.064752 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:45.067147 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:45.067152 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:45.067155 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:45.067187 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:45.069798 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:45.070524 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:45.070531 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:45.070534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:45.070674 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:45.071052 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:45.071059 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:45.071062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:45.071393 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:45.071680 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:45.072020 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 12:56:45.072244 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:56:45.076743 27136 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:56:45.083839 27136 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:56:45.121383 27659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:45.097355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51838 (local address 127.21.235.66:39989)
0809 12:56:45.108850 (+ 11495us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:45.108864 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:45.108901 (+ 37us) server_negotiation.cc:407] Connection header received
0809 12:56:45.109029 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:45.109041 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:45.109418 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:45.109668 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:45.111058 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.113348 (+ 2290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.117868 (+ 4520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.119762 (+ 1894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.119898 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:45.120142 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:45.120863 (+ 721us) server_negotiation.cc:299] Negotiation successful
0809 12:56:45.121057 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11266,"thread_start_us":12543,"threads_started":1}
I20260809 12:56:45.121431 27656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:45.096706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:39989 (local address 127.0.0.1:51838)
0809 12:56:45.108060 (+ 11354us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:45.108078 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:45.108220 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:45.110220 (+ 2000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:45.110231 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:45.110275 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:45.110774 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:45.110791 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.113564 (+ 2773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.113572 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:45.116888 (+ 3316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:45.116912 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.120298 (+ 3386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.120304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:45.120564 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:45.120632 (+ 68us) client_negotiation.cc:769] Sending connection context
0809 12:56:45.120798 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 12:56:45.121049 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11216,"thread_start_us":11048,"threads_started":1}
I20260809 12:56:45.125679 27420 tablet_service.cc:1467] Processing CreateTablet for tablet 132d8699e35c4b00a60c8ea8c3e92f5e (DEFAULT_TABLE table=test-table [id=7859cd89248f42d18f49b4755de91f0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:45.127723 27420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 132d8699e35c4b00a60c8ea8c3e92f5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:45.128101 27655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:45.096289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:36219 (local address 127.0.0.1:51042)
0809 12:56:45.106251 (+ 9962us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:45.106300 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:45.106500 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:45.119820 (+ 13320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:45.119825 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:45.119852 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:45.120435 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:45.120452 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.122785 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.122791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:45.124987 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:45.125002 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.126954 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.126967 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:45.127243 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:45.127284 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 12:56:45.127437 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 12:56:45.127587 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9694,"mutex_wait_us":33,"thread_start_us":9536,"threads_started":1}
I20260809 12:56:45.128605 27661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:45.108179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:42289 (local address 127.0.0.1:57654)
0809 12:56:45.119347 (+ 11168us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:45.119367 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:45.119506 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:45.120564 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:45.120571 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:45.120601 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:45.121234 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:45.121249 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.123286 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.123293 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:45.124804 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:45.124818 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.126971 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.126978 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:45.127925 (+ 947us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:45.127967 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 12:56:45.128066 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 12:56:45.128207 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11041,"mutex_wait_us":896,"thread_start_us":8438,"threads_started":1}
I20260809 12:56:45.128800 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:45.107234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57654 (local address 127.21.235.67:42289)
0809 12:56:45.117733 (+ 10499us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:45.117747 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:45.119675 (+ 1928us) server_negotiation.cc:407] Connection header received
0809 12:56:45.119814 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:45.119822 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:45.120057 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:45.120285 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:45.121485 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.123131 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.124960 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.126566 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.126707 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:45.127087 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:45.128224 (+ 1137us) server_negotiation.cc:299] Negotiation successful
0809 12:56:45.128405 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10242,"thread_start_us":10042,"threads_started":1}
I20260809 12:56:45.129042 27658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:45.097142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51042 (local address 127.21.235.65:36219)
0809 12:56:45.119120 (+ 21978us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:45.119128 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:45.119160 (+ 32us) server_negotiation.cc:407] Connection header received
0809 12:56:45.119275 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:45.119285 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:45.119554 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:45.120658 (+ 1104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:45.120951 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.122582 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.125153 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.126566 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.127256 (+ 690us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:45.127500 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:45.127581 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 12:56:45.127752 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21751,"thread_start_us":21532,"threads_started":1}
I20260809 12:56:45.133132 27273 tablet_service.cc:1467] Processing CreateTablet for tablet 132d8699e35c4b00a60c8ea8c3e92f5e (DEFAULT_TABLE table=test-table [id=7859cd89248f42d18f49b4755de91f0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:45.133134 27567 tablet_service.cc:1467] Processing CreateTablet for tablet 132d8699e35c4b00a60c8ea8c3e92f5e (DEFAULT_TABLE table=test-table [id=7859cd89248f42d18f49b4755de91f0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:45.134738 27273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 132d8699e35c4b00a60c8ea8c3e92f5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:45.134756 27567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 132d8699e35c4b00a60c8ea8c3e92f5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:45.139829 27662 tablet_bootstrap.cc:492] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3: Bootstrap starting.
I20260809 12:56:45.143175 27662 tablet_bootstrap.cc:654] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:45.144364 27662 log.cc:824] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:45.147202 27662 tablet_bootstrap.cc:492] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3: No bootstrap required, opened a new log
I20260809 12:56:45.147596 27662 ts_tablet_manager.cc:1397] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 12:56:45.151315 27665 tablet_bootstrap.cc:492] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e: Bootstrap starting.
I20260809 12:56:45.154971 27665 tablet_bootstrap.cc:654] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:45.156234 27665 log.cc:824] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:45.157195 27664 tablet_bootstrap.cc:492] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19: Bootstrap starting.
I20260809 12:56:45.157510 27662 raft_consensus.cc:348] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06f0fe96a80f4f86ac8a64f477e35c19" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36219 } } peers { permanent_uuid: "f7257b096c6145e3abdf36db21ca16e3" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39989 } } peers { permanent_uuid: "8348fa2c88f1462e92f633b861a3289e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42289 } }
I20260809 12:56:45.158545 27662 raft_consensus.cc:374] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:45.158675 27662 raft_consensus.cc:729] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7257b096c6145e3abdf36db21ca16e3, State: Initialized, Role: FOLLOWER
I20260809 12:56:45.159312 27662 consensus_queue.cc:260] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06f0fe96a80f4f86ac8a64f477e35c19" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36219 } } peers { permanent_uuid: "f7257b096c6145e3abdf36db21ca16e3" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39989 } } peers { permanent_uuid: "8348fa2c88f1462e92f633b861a3289e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42289 } }
I20260809 12:56:45.160765 27665 tablet_bootstrap.cc:492] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e: No bootstrap required, opened a new log
I20260809 12:56:45.161132 27665 ts_tablet_manager.cc:1397] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260809 12:56:45.161409 27664 tablet_bootstrap.cc:654] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:45.161661 27492 heartbeater.cc:503] Master 127.21.235.126:34993 was elected leader, sending a full tablet report...
I20260809 12:56:45.162379 27662 ts_tablet_manager.cc:1428] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260809 12:56:45.162879 27664 log.cc:824] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:45.166153 27664 tablet_bootstrap.cc:492] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19: No bootstrap required, opened a new log
I20260809 12:56:45.166461 27664 ts_tablet_manager.cc:1397] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 12:56:45.172982 27665 raft_consensus.cc:348] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06f0fe96a80f4f86ac8a64f477e35c19" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36219 } } peers { permanent_uuid: "f7257b096c6145e3abdf36db21ca16e3" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39989 } } peers { permanent_uuid: "8348fa2c88f1462e92f633b861a3289e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42289 } }
I20260809 12:56:45.173491 27665 raft_consensus.cc:374] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:45.173610 27665 raft_consensus.cc:729] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8348fa2c88f1462e92f633b861a3289e, State: Initialized, Role: FOLLOWER
I20260809 12:56:45.174296 27665 consensus_queue.cc:260] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06f0fe96a80f4f86ac8a64f477e35c19" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36219 } } peers { permanent_uuid: "f7257b096c6145e3abdf36db21ca16e3" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39989 } } peers { permanent_uuid: "8348fa2c88f1462e92f633b861a3289e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42289 } }
I20260809 12:56:45.175411 27640 heartbeater.cc:503] Master 127.21.235.126:34993 was elected leader, sending a full tablet report...
I20260809 12:56:45.176230 27665 ts_tablet_manager.cc:1428] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260809 12:56:45.178069 27664 raft_consensus.cc:348] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06f0fe96a80f4f86ac8a64f477e35c19" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36219 } } peers { permanent_uuid: "f7257b096c6145e3abdf36db21ca16e3" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39989 } } peers { permanent_uuid: "8348fa2c88f1462e92f633b861a3289e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42289 } }
I20260809 12:56:45.178504 27664 raft_consensus.cc:374] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:45.178643 27664 raft_consensus.cc:729] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06f0fe96a80f4f86ac8a64f477e35c19, State: Initialized, Role: FOLLOWER
I20260809 12:56:45.179275 27664 consensus_queue.cc:260] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06f0fe96a80f4f86ac8a64f477e35c19" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36219 } } peers { permanent_uuid: "f7257b096c6145e3abdf36db21ca16e3" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39989 } } peers { permanent_uuid: "8348fa2c88f1462e92f633b861a3289e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42289 } }
I20260809 12:56:45.181103 27664 ts_tablet_manager.cc:1428] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19: Time spent starting tablet: real 0.015s user 0.012s sys 0.002s
W20260809 12:56:45.201862 27348 tablet.cc:2369] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:56:45.243618 27493 tablet.cc:2369] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:56:45.269157 27641 tablet.cc:2369] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:56:45.321211 27671 raft_consensus.cc:482] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:56:45.321708 27671 raft_consensus.cc:504] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06f0fe96a80f4f86ac8a64f477e35c19" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36219 } } peers { permanent_uuid: "f7257b096c6145e3abdf36db21ca16e3" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39989 } } peers { permanent_uuid: "8348fa2c88f1462e92f633b861a3289e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42289 } }
I20260809 12:56:45.323637 27671 leader_election.cc:288] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7257b096c6145e3abdf36db21ca16e3 (127.21.235.66:39989), 8348fa2c88f1462e92f633b861a3289e (127.21.235.67:42289)
I20260809 12:56:45.332224 27659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:45.324424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:44517 (local address 127.21.235.66:39989)
0809 12:56:45.324694 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:45.324703 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:45.325542 (+ 839us) server_negotiation.cc:407] Connection header received
0809 12:56:45.325853 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:45.325858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:45.325932 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:45.325999 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:45.326863 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.327875 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.330519 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.331279 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.331453 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:45.331545 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:45.331826 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 12:56:45.331957 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260809 12:56:45.332381 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:45.323882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:39989 (local address 127.21.235.65:44517)
0809 12:56:45.325320 (+ 1438us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:45.325339 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:45.325625 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:45.326225 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:45.326228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:45.326248 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:45.326699 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:45.326713 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.328012 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.328018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:45.330282 (+ 2264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:45.330298 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.331494 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.331500 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:45.331666 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:45.331690 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 12:56:45.331798 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 12:56:45.331927 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1268,"mutex_wait_us":59,"thread_start_us":1004,"threads_started":1}
I20260809 12:56:45.333031 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:45.324168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:45531 (local address 127.21.235.67:42289)
0809 12:56:45.324371 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:45.324377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:45.325687 (+ 1310us) server_negotiation.cc:407] Connection header received
0809 12:56:45.325850 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:45.325856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:45.325932 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:45.326033 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:56:45.326696 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.327625 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.330474 (+ 2849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.331716 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.331888 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:45.331958 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:45.332648 (+ 690us) server_negotiation.cc:299] Negotiation successful
0809 12:56:45.332782 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 12:56:45.333386 27443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "132d8699e35c4b00a60c8ea8c3e92f5e" candidate_uuid: "06f0fe96a80f4f86ac8a64f477e35c19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7257b096c6145e3abdf36db21ca16e3" is_pre_election: true
I20260809 12:56:45.333773 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:45.324155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:42289 (local address 127.21.235.65:45531)
0809 12:56:45.325571 (+ 1416us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:45.325589 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:45.325712 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:45.326052 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:45.326057 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:45.326080 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:56:45.326518 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:45.326530 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.327756 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.327763 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:45.330282 (+ 2519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:45.330298 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.332130 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.332136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:45.332330 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:45.332362 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 12:56:45.333270 (+ 908us) client_negotiation.cc:240] Negotiation successful
0809 12:56:45.333415 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1297,"mutex_wait_us":97,"thread_start_us":1013,"threads_started":1}
I20260809 12:56:45.333997 27443 raft_consensus.cc:2454] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06f0fe96a80f4f86ac8a64f477e35c19 in term 0.
I20260809 12:56:45.334830 27221 leader_election.cc:302] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06f0fe96a80f4f86ac8a64f477e35c19, f7257b096c6145e3abdf36db21ca16e3; no voters:
I20260809 12:56:45.334844 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "132d8699e35c4b00a60c8ea8c3e92f5e" candidate_uuid: "06f0fe96a80f4f86ac8a64f477e35c19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8348fa2c88f1462e92f633b861a3289e" is_pre_election: true
I20260809 12:56:45.335289 27671 raft_consensus.cc:2781] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:56:45.335454 27671 raft_consensus.cc:482] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:56:45.335377 27588 raft_consensus.cc:2454] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06f0fe96a80f4f86ac8a64f477e35c19 in term 0.
I20260809 12:56:45.335583 27671 raft_consensus.cc:3037] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:45.338773 27671 raft_consensus.cc:504] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06f0fe96a80f4f86ac8a64f477e35c19" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36219 } } peers { permanent_uuid: "f7257b096c6145e3abdf36db21ca16e3" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39989 } } peers { permanent_uuid: "8348fa2c88f1462e92f633b861a3289e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42289 } }
I20260809 12:56:45.339462 27671 leader_election.cc:288] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [CANDIDATE]: Term 1 election: Requested vote from peers f7257b096c6145e3abdf36db21ca16e3 (127.21.235.66:39989), 8348fa2c88f1462e92f633b861a3289e (127.21.235.67:42289)
I20260809 12:56:45.339898 27443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "132d8699e35c4b00a60c8ea8c3e92f5e" candidate_uuid: "06f0fe96a80f4f86ac8a64f477e35c19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7257b096c6145e3abdf36db21ca16e3"
I20260809 12:56:45.339936 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "132d8699e35c4b00a60c8ea8c3e92f5e" candidate_uuid: "06f0fe96a80f4f86ac8a64f477e35c19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8348fa2c88f1462e92f633b861a3289e"
I20260809 12:56:45.340117 27443 raft_consensus.cc:3037] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:45.340112 27588 raft_consensus.cc:3037] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:45.342501 27443 raft_consensus.cc:2454] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06f0fe96a80f4f86ac8a64f477e35c19 in term 1.
I20260809 12:56:45.342885 27221 leader_election.cc:302] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06f0fe96a80f4f86ac8a64f477e35c19, f7257b096c6145e3abdf36db21ca16e3; no voters:
I20260809 12:56:45.343232 27671 raft_consensus.cc:2781] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:45.343263 27588 raft_consensus.cc:2454] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06f0fe96a80f4f86ac8a64f477e35c19 in term 1.
I20260809 12:56:45.343858 27671 raft_consensus.cc:686] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [term 1 LEADER]: Becoming Leader. State: Replica: 06f0fe96a80f4f86ac8a64f477e35c19, State: Running, Role: LEADER
I20260809 12:56:45.344652 27671 consensus_queue.cc:237] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06f0fe96a80f4f86ac8a64f477e35c19" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36219 } } peers { permanent_uuid: "f7257b096c6145e3abdf36db21ca16e3" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39989 } } peers { permanent_uuid: "8348fa2c88f1462e92f633b861a3289e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42289 } }
I20260809 12:56:45.349155 27136 catalog_manager.cc:5495] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06f0fe96a80f4f86ac8a64f477e35c19 (127.21.235.65). New cstate: current_term: 1 leader_uuid: "06f0fe96a80f4f86ac8a64f477e35c19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06f0fe96a80f4f86ac8a64f477e35c19" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 36219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7257b096c6145e3abdf36db21ca16e3" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8348fa2c88f1462e92f633b861a3289e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42289 } health_report { overall_health: UNKNOWN } } }
I20260809 12:56:45.404467 27658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:45.399615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51058 (local address 127.21.235.65:36219)
0809 12:56:45.399806 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:45.399811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:45.399829 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:56:45.399986 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:45.399989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:45.400075 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:45.400210 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:56:45.400523 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.401260 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.402546 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.402946 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.403038 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:45.403484 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:56:45.404009 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:56:45.404121 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:45.404186 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 12:56:45.404250 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 12:56:45.413182 27588 raft_consensus.cc:1261] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e [term 1 FOLLOWER]: Refusing update from remote peer 06f0fe96a80f4f86ac8a64f477e35c19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:45.413180 27443 raft_consensus.cc:1261] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3 [term 1 FOLLOWER]: Refusing update from remote peer 06f0fe96a80f4f86ac8a64f477e35c19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:45.414532 27671 consensus_queue.cc:1035] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8348fa2c88f1462e92f633b861a3289e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:56:45.414866 27674 consensus_queue.cc:1035] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7257b096c6145e3abdf36db21ca16e3" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:56:45.426646 27677 mvcc.cc:205] Tried to move back new op lower bound from 7316603721360699392 to 7316603721095561216. Current Snapshot: MvccSnapshot[applied={T|T < 7316603721360699392}]
I20260809 12:56:45.427263 27679 mvcc.cc:205] Tried to move back new op lower bound from 7316603721360699392 to 7316603721095561216. Current Snapshot: MvccSnapshot[applied={T|T < 7316603721360699392}]
I20260809 12:56:45.441427 27135 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:56:45.441740 27135 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:56:45.445741 27135 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ad48c7c97584c379f68551f8f6668a0: Sending DeleteTablet for 3 replicas of tablet 132d8699e35c4b00a60c8ea8c3e92f5e
I20260809 12:56:45.446825 27273 tablet_service.cc:1514] Processing DeleteTablet for tablet 132d8699e35c4b00a60c8ea8c3e92f5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:45 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:51042
I20260809 12:56:45.446888 27567 tablet_service.cc:1514] Processing DeleteTablet for tablet 132d8699e35c4b00a60c8ea8c3e92f5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:45 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:57654
I20260809 12:56:45.446906 27420 tablet_service.cc:1514] Processing DeleteTablet for tablet 132d8699e35c4b00a60c8ea8c3e92f5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:45 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:51838
I20260809 12:56:45.447778 27688 tablet_replica.cc:330] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3: stopping tablet replica
I20260809 12:56:45.448262 27688 raft_consensus.cc:2229] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:56:45.448818 27688 raft_consensus.cc:2258] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:45.450832 27688 ts_tablet_manager.cc:1905] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:45.452309 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 27206
I20260809 12:56:45.457957 27688 ts_tablet_manager.cc:1918] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:56:45.458142 27688 log.cc:1196] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/132d8699e35c4b00a60c8ea8c3e92f5e
I20260809 12:56:45.458747 27688 ts_tablet_manager.cc:1939] T 132d8699e35c4b00a60c8ea8c3e92f5e P f7257b096c6145e3abdf36db21ca16e3: Deleting consensus metadata
I20260809 12:56:45.460359 27686 tablet_replica.cc:330] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e: stopping tablet replica
I20260809 12:56:45.460948 27120 catalog_manager.cc:4841] TS f7257b096c6145e3abdf36db21ca16e3 (127.21.235.66:39989): tablet 132d8699e35c4b00a60c8ea8c3e92f5e (table test-table [id=7859cd89248f42d18f49b4755de91f0a]) successfully deleted
I20260809 12:56:45.460960 27686 raft_consensus.cc:2229] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:56:45.461583 27686 raft_consensus.cc:2258] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:45.463960 27686 ts_tablet_manager.cc:1905] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:45.467693 27687 tablet_replica.cc:330] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19: stopping tablet replica
I20260809 12:56:45.468218 27687 raft_consensus.cc:2229] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:56:45.469074 27687 raft_consensus.cc:2258] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:45.471264 27687 ts_tablet_manager.cc:1905] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:45.471343 27686 ts_tablet_manager.cc:1918] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:56:45.471593 27686 log.cc:1196] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/132d8699e35c4b00a60c8ea8c3e92f5e
I20260809 12:56:45.472141 27686 ts_tablet_manager.cc:1939] T 132d8699e35c4b00a60c8ea8c3e92f5e P 8348fa2c88f1462e92f633b861a3289e: Deleting consensus metadata
I20260809 12:56:45.474306 27121 catalog_manager.cc:4841] TS 8348fa2c88f1462e92f633b861a3289e (127.21.235.67:42289): tablet 132d8699e35c4b00a60c8ea8c3e92f5e (table test-table [id=7859cd89248f42d18f49b4755de91f0a]) successfully deleted
Aug 09 12:56:45 dist-test-slave-gp6n krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 12:56:45.479060 27687 ts_tablet_manager.cc:1918] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:56:45.479254 27687 log.cc:1196] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/132d8699e35c4b00a60c8ea8c3e92f5e
I20260809 12:56:45.479832 27687 ts_tablet_manager.cc:1939] T 132d8699e35c4b00a60c8ea8c3e92f5e P 06f0fe96a80f4f86ac8a64f477e35c19: Deleting consensus metadata
I20260809 12:56:45.481556 27121 catalog_manager.cc:4841] TS 06f0fe96a80f4f86ac8a64f477e35c19 (127.21.235.65:36219): tablet 132d8699e35c4b00a60c8ea8c3e92f5e (table test-table [id=7859cd89248f42d18f49b4755de91f0a]) successfully deleted
I20260809 12:56:45.483917 27658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:45.453876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51060 (local address 127.21.235.65:36219)
0809 12:56:45.454622 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:45.454631 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:45.466934 (+ 12303us) server_negotiation.cc:407] Connection header received
0809 12:56:45.467129 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:45.467138 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:45.467266 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:45.467379 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:45.469187 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.470453 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.472413 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.473125 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.473230 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:45.477558 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:45.477569 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:45.477582 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:45.477623 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:45.480911 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:45.481691 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:45.481701 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:45.481707 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:45.481875 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:45.482267 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:45.482272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:45.482276 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:45.482687 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:45.483008 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:45.483280 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 12:56:45.483528 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585}
I20260809 12:56:45.484583 27342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:45.586724 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 27206
I20260809 12:56:45.607926 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 27354
Aug 09 12:56:45 dist-test-slave-gp6n krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.66@KRBTEST.COM
I20260809 12:56:45.623937 27659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:45.608754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51848 (local address 127.21.235.66:39989)
0809 12:56:45.609025 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:45.609033 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:45.609056 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:56:45.609302 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:45.609311 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:45.609398 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:45.609537 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:45.611040 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.611903 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.613311 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.613756 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.613909 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:45.618472 (+ 4563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:45.618487 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:45.618503 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:45.618538 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:45.621154 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:45.621993 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:45.622001 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:45.622003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:45.622132 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:45.622517 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:45.622520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:45.622522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:45.622888 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:45.623118 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:45.623374 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 12:56:45.623552 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260809 12:56:45.624637 27486 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:45.713580 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 27354
I20260809 12:56:45.733093 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 27499
Aug 09 12:56:45 dist-test-slave-gp6n krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.67@KRBTEST.COM
I20260809 12:56:45.748829 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:45.733882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57664 (local address 127.21.235.67:42289)
0809 12:56:45.734086 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:45.734094 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:45.734120 (+ 26us) server_negotiation.cc:407] Connection header received
0809 12:56:45.734347 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:45.734352 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:45.734479 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:45.734606 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:45.735925 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.736660 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.737946 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:45.738377 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:45.738551 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:45.743047 (+ 4496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:45.743062 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:45.743079 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:45.743114 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:45.745904 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:45.746711 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:45.746719 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:45.746721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:45.746876 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:45.747259 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:45.747266 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:45.747269 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:45.747630 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:45.747937 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:45.748247 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 12:56:45.748478 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:56:45.749524 27634 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:45.839653 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 27499
I20260809 12:56:45.859707 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 27100
I20260809 12:56:45.860848 27169 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:45.970621 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 27100
2026-08-09T12:56:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4462 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T12:56:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:56:46Z Disabled control of system clock
I20260809 12:56:46.045684 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:36203
--webserver_interface=127.21.235.126
--webserver_port=0
--builtin_ntp_servers=127.21.235.84:35901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:36203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 12:56:46.392686 27698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:46.393006 27698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:46.393167 27698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:46.400377 27698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:56:46.400485 27698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:46.400553 27698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:46.400647 27698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:56:46.400728 27698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:56:46.409938 27698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:35901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:36203
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:36203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:46.411345 27698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:46.413175 27698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:46.426287 27703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:46.426205 27706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:46.426262 27704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:46.427124 27698 server_base.cc:1034] running on GCE node
I20260809 12:56:46.427855 27698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:46.429327 27698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:46.430514 27698 hybrid_clock.cc:648] HybridClock initialized: now 1786280206430467 us; error 35 us; skew 500 ppm
I20260809 12:56:46.430889 27698 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:56:46.433116 27698 webserver.cc:460] Webserver started at http://127.21.235.126:37827/ using document root <none> and password file <none>
I20260809 12:56:46.433844 27698 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:46.433965 27698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:46.434315 27698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:46.437335 27698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "a1ee1f71e7f244af92bbc341a6e3bf3c"
format_stamp: "Formatted at 2026-08-09 12:56:46 on dist-test-slave-gp6n"
server_key: "940c046b534c2f4f3d07080651984c78"
server_key_iv: "584ddb271bdb351b6cb99a541e880702"
server_key_version: "encryptionkey@0"
I20260809 12:56:46.438220 27698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "a1ee1f71e7f244af92bbc341a6e3bf3c"
format_stamp: "Formatted at 2026-08-09 12:56:46 on dist-test-slave-gp6n"
server_key: "940c046b534c2f4f3d07080651984c78"
server_key_iv: "584ddb271bdb351b6cb99a541e880702"
server_key_version: "encryptionkey@0"
I20260809 12:56:46.443424 27698 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:56:46.447022 27712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:46.448480 27698 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:56:46.448671 27698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "a1ee1f71e7f244af92bbc341a6e3bf3c"
format_stamp: "Formatted at 2026-08-09 12:56:46 on dist-test-slave-gp6n"
server_key: "940c046b534c2f4f3d07080651984c78"
server_key_iv: "584ddb271bdb351b6cb99a541e880702"
server_key_version: "encryptionkey@0"
I20260809 12:56:46.448928 27698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:46.454972 27698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:46.455765 27698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:46.456082 27698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:46.474670 27698 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:36203
I20260809 12:56:46.474704 27771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:36203 every 8 connection(s)
I20260809 12:56:46.476085 27698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:56:46.479945 27772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:46.486600 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 27698
I20260809 12:56:46.486792 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:56:46.487466 22445 external_mini_cluster.cc:1428] Setting key be262e4179660565172d222c7bb26652
I20260809 12:56:46.491180 27772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c: Bootstrap starting.
I20260809 12:56:46.495553 27772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:46.496944 27772 log.cc:824] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:46.497608 27775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:46.489641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37664 (local address 127.21.235.126:36203)
0809 12:56:46.490768 (+ 1127us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:46.490785 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:46.490840 (+ 55us) server_negotiation.cc:407] Connection header received
0809 12:56:46.493319 (+ 2479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:46.493357 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:46.493922 (+ 565us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:46.494372 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:46.495543 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:46.495552 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:46.495569 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:46.495591 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:46.495680 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:46.495703 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:46.495907 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:46.496084 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 12:56:46.496366 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":330,"threads_started":1}
I20260809 12:56:46.500036 27772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c: No bootstrap required, opened a new log
I20260809 12:56:46.507117 27772 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1ee1f71e7f244af92bbc341a6e3bf3c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36203 } }
I20260809 12:56:46.507475 27772 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:46.507586 27772 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1ee1f71e7f244af92bbc341a6e3bf3c, State: Initialized, Role: FOLLOWER
I20260809 12:56:46.508175 27772 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1ee1f71e7f244af92bbc341a6e3bf3c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36203 } }
I20260809 12:56:46.508404 27772 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:56:46.508548 27772 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:56:46.508742 27772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:46.511109 27772 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1ee1f71e7f244af92bbc341a6e3bf3c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36203 } }
I20260809 12:56:46.511616 27772 leader_election.cc:302] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1ee1f71e7f244af92bbc341a6e3bf3c; no voters:
I20260809 12:56:46.512217 27772 leader_election.cc:288] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:56:46.512377 27778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:46.513129 27778 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c [term 1 LEADER]: Becoming Leader. State: Replica: a1ee1f71e7f244af92bbc341a6e3bf3c, State: Running, Role: LEADER
I20260809 12:56:46.513734 27778 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1ee1f71e7f244af92bbc341a6e3bf3c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36203 } }
I20260809 12:56:46.514340 27772 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:56:46.516513 27779 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1ee1f71e7f244af92bbc341a6e3bf3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1ee1f71e7f244af92bbc341a6e3bf3c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36203 } } }
I20260809 12:56:46.516878 27779 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:46.517441 27780 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1ee1f71e7f244af92bbc341a6e3bf3c. Latest consensus state: current_term: 1 leader_uuid: "a1ee1f71e7f244af92bbc341a6e3bf3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1ee1f71e7f244af92bbc341a6e3bf3c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36203 } } }
I20260809 12:56:46.517764 27780 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:46.523976 27785 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:56:46.528401 27785 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:56:46.536731 27785 catalog_manager.cc:1347] Generated new cluster ID: 5ce2da18922f40b3912ee0d30f735ce5
I20260809 12:56:46.536854 27785 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:56:46.555593 27785 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:56:46.556032 27785 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:56:46.567610 27785 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c: Generated new TSK 0
I20260809 12:56:46.568430 27785 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:56:46.575503 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:36203
--builtin_ntp_servers=127.21.235.84:35901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 12:56:46.911988 27799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:46.912283 27799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:46.912375 27799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:46.919977 27799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:46.920084 27799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:46.920197 27799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:56:46.929509 27799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:35901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:36203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:46.930836 27799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:46.932682 27799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:46.946269 27807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:46.946482 27804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:46.946240 27805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:46.947129 27799 server_base.cc:1034] running on GCE node
I20260809 12:56:46.947784 27799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:46.948792 27799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:46.950052 27799 hybrid_clock.cc:648] HybridClock initialized: now 1786280206950008 us; error 50 us; skew 500 ppm
I20260809 12:56:46.950452 27799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:56:46.952893 27799 webserver.cc:460] Webserver started at http://127.21.235.65:46347/ using document root <none> and password file <none>
I20260809 12:56:46.953758 27799 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:46.953855 27799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:46.954207 27799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:46.957265 27799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "073fe93a027145bfaaa6d07df7ad638f"
format_stamp: "Formatted at 2026-08-09 12:56:46 on dist-test-slave-gp6n"
server_key: "23a4f0294cddb85090189fc71ed8e7b4"
server_key_iv: "0afb51de6d32e88bcc5986a9c943dd72"
server_key_version: "encryptionkey@0"
I20260809 12:56:46.958132 27799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "073fe93a027145bfaaa6d07df7ad638f"
format_stamp: "Formatted at 2026-08-09 12:56:46 on dist-test-slave-gp6n"
server_key: "23a4f0294cddb85090189fc71ed8e7b4"
server_key_iv: "0afb51de6d32e88bcc5986a9c943dd72"
server_key_version: "encryptionkey@0"
I20260809 12:56:46.963532 27799 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:56:46.967098 27814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:46.968436 27799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:56:46.968612 27799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "073fe93a027145bfaaa6d07df7ad638f"
format_stamp: "Formatted at 2026-08-09 12:56:46 on dist-test-slave-gp6n"
server_key: "23a4f0294cddb85090189fc71ed8e7b4"
server_key_iv: "0afb51de6d32e88bcc5986a9c943dd72"
server_key_version: "encryptionkey@0"
I20260809 12:56:46.968858 27799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:46.974709 27799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:46.975512 27799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:46.975793 27799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:46.976802 27799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:46.978240 27799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:46.978365 27799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:46.978524 27799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:46.978618 27799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:47.012494 27799 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:46451
I20260809 12:56:47.012554 27932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:46451 every 8 connection(s)
I20260809 12:56:47.013872 27799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:56:47.018172 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 27799
I20260809 12:56:47.018368 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:56:47.019122 22445 external_mini_cluster.cc:1428] Setting key 098eda0366f7927aba32b5ed34f2cd9e
I20260809 12:56:47.021942 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:36203
--builtin_ntp_servers=127.21.235.84:35901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 12:56:47.023425 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:47.016691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:49751 (local address 127.21.235.126:36203)
0809 12:56:47.017145 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:47.017151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:47.018599 (+ 1448us) server_negotiation.cc:407] Connection header received
0809 12:56:47.020785 (+ 2186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:47.020790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:47.020858 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:47.020963 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:47.022645 (+ 1682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:47.022651 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:47.022656 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:47.022675 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:47.022725 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:47.022737 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:47.022870 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:47.023064 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 12:56:47.023133 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":216,"threads_started":1}
I20260809 12:56:47.024682 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:47.017379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:36203 (local address 127.21.235.65:49751)
0809 12:56:47.018369 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:47.018464 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:47.020522 (+ 2058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:47.021421 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:47.021429 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:47.021518 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:47.022295 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:56:47.022297 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:47.022334 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:56:47.022482 (+ 148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:47.022838 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:47.022851 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:47.022859 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 12:56:47.023153 (+ 294us) client_negotiation.cc:240] Negotiation successful
0809 12:56:47.023428 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":277,"threads_started":1}
I20260809 12:56:47.026209 27933 heartbeater.cc:347] Connected to a master server at 127.21.235.126:36203
I20260809 12:56:47.026563 27933 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:47.027281 27933 heartbeater.cc:511] Master 127.21.235.126:36203 requested a full tablet report, sending...
I20260809 12:56:47.029388 27730 ts_manager.cc:194] Registered new tserver with Master: 073fe93a027145bfaaa6d07df7ad638f (127.21.235.65:46451)
W20260809 12:56:47.373309 27938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:47.373692 27938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:47.373852 27938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:47.381125 27938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:47.381263 27938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:47.381440 27938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:56:47.390583 27938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:35901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:36203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:47.391911 27938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:47.393690 27938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:47.406778 27943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:47.406939 27944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:47.407246 27946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:47.407884 27938 server_base.cc:1034] running on GCE node
I20260809 12:56:47.408488 27938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:47.409550 27938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:47.410789 27938 hybrid_clock.cc:648] HybridClock initialized: now 1786280207410738 us; error 57 us; skew 500 ppm
I20260809 12:56:47.411135 27938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:56:47.413542 27938 webserver.cc:460] Webserver started at http://127.21.235.66:45333/ using document root <none> and password file <none>
I20260809 12:56:47.414347 27938 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:47.414512 27938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:47.414880 27938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:47.417840 27938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "bfdc99c22b6c4564ab8571edfc3291ee"
format_stamp: "Formatted at 2026-08-09 12:56:47 on dist-test-slave-gp6n"
server_key: "3e48287b2ae78e8caec4ab013dc36861"
server_key_iv: "7c09dc673aba93b788ec37b490ca4c9b"
server_key_version: "encryptionkey@0"
I20260809 12:56:47.418716 27938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "bfdc99c22b6c4564ab8571edfc3291ee"
format_stamp: "Formatted at 2026-08-09 12:56:47 on dist-test-slave-gp6n"
server_key: "3e48287b2ae78e8caec4ab013dc36861"
server_key_iv: "7c09dc673aba93b788ec37b490ca4c9b"
server_key_version: "encryptionkey@0"
I20260809 12:56:47.424351 27938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 12:56:47.427970 27953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:47.429441 27938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:56:47.429631 27938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "bfdc99c22b6c4564ab8571edfc3291ee"
format_stamp: "Formatted at 2026-08-09 12:56:47 on dist-test-slave-gp6n"
server_key: "3e48287b2ae78e8caec4ab013dc36861"
server_key_iv: "7c09dc673aba93b788ec37b490ca4c9b"
server_key_version: "encryptionkey@0"
I20260809 12:56:47.429862 27938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:47.435607 27938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:47.436405 27938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:47.436688 27938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:47.437564 27938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:47.439040 27938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:47.439167 27938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:47.439317 27938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:47.439409 27938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:47.473748 27938 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:39139
I20260809 12:56:47.473858 28079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:39139 every 8 connection(s)
I20260809 12:56:47.475219 27938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:56:47.476553 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 27938
I20260809 12:56:47.476754 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:56:47.477238 22445 external_mini_cluster.cc:1428] Setting key 1462025100cda4a684ee812b17e9424b
I20260809 12:56:47.480775 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:36203
--builtin_ntp_servers=127.21.235.84:35901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 12:56:47.485977 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:47.479028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:33677 (local address 127.21.235.126:36203)
0809 12:56:47.479227 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:47.479234 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:47.480821 (+ 1587us) server_negotiation.cc:407] Connection header received
0809 12:56:47.483314 (+ 2493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:47.483322 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:47.483395 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:47.483513 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:47.485097 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:47.485105 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:47.485112 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:47.485135 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:47.485179 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:47.485199 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:47.485353 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:47.485520 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 12:56:47.485591 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:56:47.487126 28082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:47.479555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:36203 (local address 127.21.235.66:33677)
0809 12:56:47.480678 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:47.480722 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:47.482988 (+ 2266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:47.483805 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:47.483819 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:47.483941 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:47.484739 (+ 798us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:56:47.484743 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:47.484793 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:56:47.484901 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:47.485305 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:47.485314 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:47.485322 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 12:56:47.485579 (+ 257us) client_negotiation.cc:240] Negotiation successful
0809 12:56:47.485821 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":359,"threads_started":1}
I20260809 12:56:47.488579 28080 heartbeater.cc:347] Connected to a master server at 127.21.235.126:36203
I20260809 12:56:47.488936 28080 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:47.489701 28080 heartbeater.cc:511] Master 127.21.235.126:36203 requested a full tablet report, sending...
I20260809 12:56:47.491169 27730 ts_manager.cc:194] Registered new tserver with Master: bfdc99c22b6c4564ab8571edfc3291ee (127.21.235.66:39139)
W20260809 12:56:47.819623 28083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:47.819909 28083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:47.820107 28083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:47.827687 28083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:47.827787 28083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:47.827961 28083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:56:47.837863 28083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:35901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:36203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:47.839279 28083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:47.841081 28083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:47.854691 28089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:47.854911 28090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:47.855198 28092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:47.856225 28083 server_base.cc:1034] running on GCE node
I20260809 12:56:47.856849 28083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:47.857813 28083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:47.859025 28083 hybrid_clock.cc:648] HybridClock initialized: now 1786280207858969 us; error 42 us; skew 500 ppm
I20260809 12:56:47.859325 28083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:56:47.861723 28083 webserver.cc:460] Webserver started at http://127.21.235.67:46403/ using document root <none> and password file <none>
I20260809 12:56:47.862500 28083 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:47.862601 28083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:47.862890 28083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:47.865870 28083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "a12d06067dd14c38aa086dc0ce3312df"
format_stamp: "Formatted at 2026-08-09 12:56:47 on dist-test-slave-gp6n"
server_key: "563cf20f43deb0c2d75e05f30e6ae1b4"
server_key_iv: "e2147790ca34714c61f489cba2003062"
server_key_version: "encryptionkey@0"
I20260809 12:56:47.866801 28083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "a12d06067dd14c38aa086dc0ce3312df"
format_stamp: "Formatted at 2026-08-09 12:56:47 on dist-test-slave-gp6n"
server_key: "563cf20f43deb0c2d75e05f30e6ae1b4"
server_key_iv: "e2147790ca34714c61f489cba2003062"
server_key_version: "encryptionkey@0"
I20260809 12:56:47.872217 28083 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:56:47.875815 28100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:47.877168 28083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:56:47.877334 28083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "a12d06067dd14c38aa086dc0ce3312df"
format_stamp: "Formatted at 2026-08-09 12:56:47 on dist-test-slave-gp6n"
server_key: "563cf20f43deb0c2d75e05f30e6ae1b4"
server_key_iv: "e2147790ca34714c61f489cba2003062"
server_key_version: "encryptionkey@0"
I20260809 12:56:47.877538 28083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:47.883378 28083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:47.884166 28083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:47.884438 28083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:47.885373 28083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:47.886885 28083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:47.886976 28083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:47.887143 28083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:47.887238 28083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:47.923378 28083 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:44381
I20260809 12:56:47.923513 28223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:44381 every 8 connection(s)
I20260809 12:56:47.924787 28083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:56:47.925956 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 28083
I20260809 12:56:47.926136 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:56:47.926694 22445 external_mini_cluster.cc:1428] Setting key 7c16d82569f49ae8fd742fd92440cb9e
I20260809 12:56:47.934231 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:47.928021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:56921 (local address 127.21.235.126:36203)
0809 12:56:47.928262 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:47.928269 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:47.930139 (+ 1870us) server_negotiation.cc:407] Connection header received
0809 12:56:47.932037 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:47.932041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:47.932093 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:47.932179 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:47.933608 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:47.933612 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:47.933615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:47.933630 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:47.933663 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:47.933675 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:47.933766 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:47.933964 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 12:56:47.934016 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 12:56:47.935489 28226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:47.928745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:36203 (local address 127.21.235.67:56921)
0809 12:56:47.929974 (+ 1229us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:47.930023 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:47.931781 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:47.932448 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:47.932457 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:47.932558 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:47.933289 (+ 731us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:56:47.933295 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:47.933358 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:56:47.933446 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:47.933768 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:47.933776 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:47.933782 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 12:56:47.934024 (+ 242us) client_negotiation.cc:240] Negotiation successful
0809 12:56:47.934240 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":394,"threads_started":1}
I20260809 12:56:47.936877 28224 heartbeater.cc:347] Connected to a master server at 127.21.235.126:36203
I20260809 12:56:47.937151 28224 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:47.937713 28224 heartbeater.cc:511] Master 127.21.235.126:36203 requested a full tablet report, sending...
I20260809 12:56:47.939102 27730 ts_manager.cc:194] Registered new tserver with Master: a12d06067dd14c38aa086dc0ce3312df (127.21.235.67:44381)
I20260809 12:56:47.942591 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:56:47.949885 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:47.947362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37672 (local address 127.21.235.126:36203)
0809 12:56:47.947542 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:47.947546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:47.948169 (+ 623us) server_negotiation.cc:407] Connection header received
0809 12:56:47.948313 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:47.948317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:47.948367 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:47.948447 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:47.949342 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:47.949346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:47.949349 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:47.949366 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:47.949398 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:47.949410 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:47.949524 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:47.949625 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 12:56:47.949673 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 12:56:47.954053 27730 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:56:47.960867 27730 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:56:47.981606 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:47.974110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:46451 (local address 127.0.0.1:33658)
0809 12:56:47.974966 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:47.975029 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:47.975183 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:47.976550 (+ 1367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:47.976561 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:47.976674 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:47.978522 (+ 1848us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:56:47.978527 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:47.978569 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:56:47.978676 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:47.980600 (+ 1924us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:47.980617 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:47.980630 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 12:56:47.980849 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 12:56:47.980975 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":1211,"threads_started":1}
I20260809 12:56:47.984217 28239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:47.975258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33658 (local address 127.21.235.65:46451)
0809 12:56:47.975837 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:47.975849 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:47.975888 (+ 39us) server_negotiation.cc:407] Connection header received
0809 12:56:47.975975 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:47.975984 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:47.976270 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:47.979015 (+ 2745us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:47.980261 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:47.980273 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:47.980293 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:47.980323 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:47.980392 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:47.980418 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:47.983478 (+ 3060us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:47.983598 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 12:56:47.983731 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":278,"threads_started":1}
I20260809 12:56:47.988045 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:47.974160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:39139 (local address 127.0.0.1:50450)
0809 12:56:47.975090 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:47.975113 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:47.975214 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:47.985980 (+ 10766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:47.985985 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:47.986011 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:47.986785 (+ 774us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:56:47.986789 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:47.986807 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:56:47.986863 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:47.987423 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:47.987432 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:47.987437 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 12:56:47.987570 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 12:56:47.987626 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"thread_start_us":554,"threads_started":1}
I20260809 12:56:47.988842 28238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:47.974848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50450 (local address 127.21.235.66:39139)
0809 12:56:47.985562 (+ 10714us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:47.985569 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:47.985593 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:56:47.985648 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:47.985654 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:47.985803 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:47.985997 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:47.987144 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:47.987151 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:47.987161 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:47.987173 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:47.987245 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:47.987266 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:47.988498 (+ 1232us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:47.988551 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 12:56:47.988633 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10573,"threads_started":1}
I20260809 12:56:47.989416 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:47.975633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:44381 (local address 127.0.0.1:50892)
0809 12:56:47.981843 (+ 6210us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:47.981860 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:47.981935 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:47.985989 (+ 4054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:47.985994 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:47.986039 (+ 45us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:47.987495 (+ 1456us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:56:47.987499 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:47.987516 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:56:47.987562 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:47.989011 (+ 1449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:47.989016 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:47.989021 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 12:56:47.989096 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 12:56:47.989145 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6122,"thread_start_us":9059,"threads_started":1}
I20260809 12:56:47.989399 27864 tablet_service.cc:1467] Processing CreateTablet for tablet 59929299a4e24dd5a2d9069cf514c355 (DEFAULT_TABLE table=test-table [id=aa58347e5ed549d0936f4cab07ec8def]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:47.990267 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:47.976025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50892 (local address 127.21.235.67:44381)
0809 12:56:47.985423 (+ 9398us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:47.985432 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:47.985468 (+ 36us) server_negotiation.cc:407] Connection header received
0809 12:56:47.985545 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:47.985553 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:47.985792 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:47.987729 (+ 1937us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:47.988767 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:47.988775 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:47.988787 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:47.988811 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:47.988861 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:47.988879 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:47.989727 (+ 848us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:47.989801 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 12:56:47.989893 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9217,"thread_start_us":9021,"threads_started":1}
I20260809 12:56:47.991058 27864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59929299a4e24dd5a2d9069cf514c355. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:47.993188 28007 tablet_service.cc:1467] Processing CreateTablet for tablet 59929299a4e24dd5a2d9069cf514c355 (DEFAULT_TABLE table=test-table [id=aa58347e5ed549d0936f4cab07ec8def]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:47.993870 28153 tablet_service.cc:1467] Processing CreateTablet for tablet 59929299a4e24dd5a2d9069cf514c355 (DEFAULT_TABLE table=test-table [id=aa58347e5ed549d0936f4cab07ec8def]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:56:47.994566 28007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59929299a4e24dd5a2d9069cf514c355. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:47.995256 28153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59929299a4e24dd5a2d9069cf514c355. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:48.002754 28243 tablet_bootstrap.cc:492] T 59929299a4e24dd5a2d9069cf514c355 P 073fe93a027145bfaaa6d07df7ad638f: Bootstrap starting.
I20260809 12:56:48.006596 28243 tablet_bootstrap.cc:654] T 59929299a4e24dd5a2d9069cf514c355 P 073fe93a027145bfaaa6d07df7ad638f: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:48.007642 28243 log.cc:824] T 59929299a4e24dd5a2d9069cf514c355 P 073fe93a027145bfaaa6d07df7ad638f: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:48.010032 28245 tablet_bootstrap.cc:492] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee: Bootstrap starting.
I20260809 12:56:48.010785 28243 tablet_bootstrap.cc:492] T 59929299a4e24dd5a2d9069cf514c355 P 073fe93a027145bfaaa6d07df7ad638f: No bootstrap required, opened a new log
I20260809 12:56:48.011145 28243 ts_tablet_manager.cc:1397] T 59929299a4e24dd5a2d9069cf514c355 P 073fe93a027145bfaaa6d07df7ad638f: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260809 12:56:48.011770 28246 tablet_bootstrap.cc:492] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df: Bootstrap starting.
I20260809 12:56:48.013846 28245 tablet_bootstrap.cc:654] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:48.015187 28246 tablet_bootstrap.cc:654] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:48.016165 28245 log.cc:824] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:48.016606 28246 log.cc:824] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:48.024296 28246 tablet_bootstrap.cc:492] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df: No bootstrap required, opened a new log
I20260809 12:56:48.024735 28246 ts_tablet_manager.cc:1397] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.003s
I20260809 12:56:48.027480 28243 raft_consensus.cc:348] T 59929299a4e24dd5a2d9069cf514c355 P 073fe93a027145bfaaa6d07df7ad638f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073fe93a027145bfaaa6d07df7ad638f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46451 } } peers { permanent_uuid: "bfdc99c22b6c4564ab8571edfc3291ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39139 } } peers { permanent_uuid: "a12d06067dd14c38aa086dc0ce3312df" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44381 } }
I20260809 12:56:48.028090 28243 raft_consensus.cc:374] T 59929299a4e24dd5a2d9069cf514c355 P 073fe93a027145bfaaa6d07df7ad638f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:48.028245 28243 raft_consensus.cc:729] T 59929299a4e24dd5a2d9069cf514c355 P 073fe93a027145bfaaa6d07df7ad638f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073fe93a027145bfaaa6d07df7ad638f, State: Initialized, Role: FOLLOWER
I20260809 12:56:48.029119 28243 consensus_queue.cc:260] T 59929299a4e24dd5a2d9069cf514c355 P 073fe93a027145bfaaa6d07df7ad638f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073fe93a027145bfaaa6d07df7ad638f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46451 } } peers { permanent_uuid: "bfdc99c22b6c4564ab8571edfc3291ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39139 } } peers { permanent_uuid: "a12d06067dd14c38aa086dc0ce3312df" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44381 } }
I20260809 12:56:48.030356 27933 heartbeater.cc:503] Master 127.21.235.126:36203 was elected leader, sending a full tablet report...
I20260809 12:56:48.031092 28243 ts_tablet_manager.cc:1428] T 59929299a4e24dd5a2d9069cf514c355 P 073fe93a027145bfaaa6d07df7ad638f: Time spent starting tablet: real 0.020s user 0.013s sys 0.001s
I20260809 12:56:48.031364 28245 tablet_bootstrap.cc:492] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee: No bootstrap required, opened a new log
I20260809 12:56:48.031677 28245 ts_tablet_manager.cc:1397] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.011s
I20260809 12:56:48.036092 28246 raft_consensus.cc:348] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073fe93a027145bfaaa6d07df7ad638f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46451 } } peers { permanent_uuid: "bfdc99c22b6c4564ab8571edfc3291ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39139 } } peers { permanent_uuid: "a12d06067dd14c38aa086dc0ce3312df" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44381 } }
I20260809 12:56:48.036541 28246 raft_consensus.cc:374] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:48.036636 28246 raft_consensus.cc:729] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a12d06067dd14c38aa086dc0ce3312df, State: Initialized, Role: FOLLOWER
I20260809 12:56:48.037266 28246 consensus_queue.cc:260] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073fe93a027145bfaaa6d07df7ad638f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46451 } } peers { permanent_uuid: "bfdc99c22b6c4564ab8571edfc3291ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39139 } } peers { permanent_uuid: "a12d06067dd14c38aa086dc0ce3312df" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44381 } }
I20260809 12:56:48.038964 28224 heartbeater.cc:503] Master 127.21.235.126:36203 was elected leader, sending a full tablet report...
I20260809 12:56:48.039217 28246 ts_tablet_manager.cc:1428] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df: Time spent starting tablet: real 0.014s user 0.004s sys 0.009s
I20260809 12:56:48.041652 28245 raft_consensus.cc:348] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073fe93a027145bfaaa6d07df7ad638f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46451 } } peers { permanent_uuid: "bfdc99c22b6c4564ab8571edfc3291ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39139 } } peers { permanent_uuid: "a12d06067dd14c38aa086dc0ce3312df" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44381 } }
I20260809 12:56:48.042106 28245 raft_consensus.cc:374] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:48.042229 28245 raft_consensus.cc:729] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfdc99c22b6c4564ab8571edfc3291ee, State: Initialized, Role: FOLLOWER
I20260809 12:56:48.042814 28245 consensus_queue.cc:260] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073fe93a027145bfaaa6d07df7ad638f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46451 } } peers { permanent_uuid: "bfdc99c22b6c4564ab8571edfc3291ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39139 } } peers { permanent_uuid: "a12d06067dd14c38aa086dc0ce3312df" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44381 } }
I20260809 12:56:48.043856 28080 heartbeater.cc:503] Master 127.21.235.126:36203 was elected leader, sending a full tablet report...
I20260809 12:56:48.044376 28245 ts_tablet_manager.cc:1428] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260809 12:56:48.073248 28252 raft_consensus.cc:482] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:56:48.073628 28252 raft_consensus.cc:504] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073fe93a027145bfaaa6d07df7ad638f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46451 } } peers { permanent_uuid: "bfdc99c22b6c4564ab8571edfc3291ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39139 } } peers { permanent_uuid: "a12d06067dd14c38aa086dc0ce3312df" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44381 } }
I20260809 12:56:48.075472 28252 leader_election.cc:288] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 073fe93a027145bfaaa6d07df7ad638f (127.21.235.65:46451), a12d06067dd14c38aa086dc0ce3312df (127.21.235.67:44381)
I20260809 12:56:48.078687 28239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:48.075789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:59231 (local address 127.21.235.65:46451)
0809 12:56:48.076037 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:48.076046 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:48.076483 (+ 437us) server_negotiation.cc:407] Connection header received
0809 12:56:48.076667 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:48.076674 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:48.076747 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:48.076860 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:48.077968 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:48.077974 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:48.077979 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:48.078013 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:48.078044 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:48.078056 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:48.078180 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:48.078303 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 12:56:48.078374 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 12:56:48.078889 28253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:48.075691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:46451 (local address 127.21.235.66:59231)
0809 12:56:48.076369 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:48.076388 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:48.076525 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:48.076914 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:48.076919 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:48.076951 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:48.077767 (+ 816us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:56:48.077770 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:48.077784 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:56:48.077825 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:48.078159 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:48.078164 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:48.078169 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 12:56:48.078351 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 12:56:48.078426 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":352,"threads_started":1}
I20260809 12:56:48.079056 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:48.076609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:40171 (local address 127.21.235.67:44381)
0809 12:56:48.076764 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:48.076768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:48.077195 (+ 427us) server_negotiation.cc:407] Connection header received
0809 12:56:48.077277 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:48.077285 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:48.077351 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:48.077447 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:48.078428 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:48.078434 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:48.078438 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:48.078458 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:48.078493 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:48.078504 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:48.078637 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:48.078710 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 12:56:48.078774 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 12:56:48.079222 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:48.076370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:44381 (local address 127.21.235.66:40171)
0809 12:56:48.077097 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:48.077111 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:48.077196 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:48.077503 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:48.077508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:48.077535 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:48.078212 (+ 677us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:56:48.078214 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:48.078225 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:56:48.078258 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:48.078599 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:48.078604 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:48.078609 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 12:56:48.078760 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 12:56:48.078811 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":401,"threads_started":1}
I20260809 12:56:48.080096 28176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59929299a4e24dd5a2d9069cf514c355" candidate_uuid: "bfdc99c22b6c4564ab8571edfc3291ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a12d06067dd14c38aa086dc0ce3312df" is_pre_election: true
I20260809 12:56:48.080103 27885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59929299a4e24dd5a2d9069cf514c355" candidate_uuid: "bfdc99c22b6c4564ab8571edfc3291ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073fe93a027145bfaaa6d07df7ad638f" is_pre_election: true
I20260809 12:56:48.080614 27885 raft_consensus.cc:2454] T 59929299a4e24dd5a2d9069cf514c355 P 073fe93a027145bfaaa6d07df7ad638f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfdc99c22b6c4564ab8571edfc3291ee in term 0.
I20260809 12:56:48.080613 28176 raft_consensus.cc:2454] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfdc99c22b6c4564ab8571edfc3291ee in term 0.
I20260809 12:56:48.081265 27955 leader_election.cc:302] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a12d06067dd14c38aa086dc0ce3312df, bfdc99c22b6c4564ab8571edfc3291ee; no voters:
I20260809 12:56:48.081681 28252 raft_consensus.cc:2781] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:56:48.081820 28252 raft_consensus.cc:482] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:56:48.081930 28252 raft_consensus.cc:3037] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:48.084265 28252 raft_consensus.cc:504] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073fe93a027145bfaaa6d07df7ad638f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46451 } } peers { permanent_uuid: "bfdc99c22b6c4564ab8571edfc3291ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39139 } } peers { permanent_uuid: "a12d06067dd14c38aa086dc0ce3312df" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44381 } }
I20260809 12:56:48.084775 28252 leader_election.cc:288] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [CANDIDATE]: Term 1 election: Requested vote from peers 073fe93a027145bfaaa6d07df7ad638f (127.21.235.65:46451), a12d06067dd14c38aa086dc0ce3312df (127.21.235.67:44381)
I20260809 12:56:48.085232 27885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59929299a4e24dd5a2d9069cf514c355" candidate_uuid: "bfdc99c22b6c4564ab8571edfc3291ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073fe93a027145bfaaa6d07df7ad638f"
I20260809 12:56:48.085314 28176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59929299a4e24dd5a2d9069cf514c355" candidate_uuid: "bfdc99c22b6c4564ab8571edfc3291ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a12d06067dd14c38aa086dc0ce3312df"
I20260809 12:56:48.085422 27885 raft_consensus.cc:3037] T 59929299a4e24dd5a2d9069cf514c355 P 073fe93a027145bfaaa6d07df7ad638f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:48.085572 28176 raft_consensus.cc:3037] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:48.087774 27885 raft_consensus.cc:2454] T 59929299a4e24dd5a2d9069cf514c355 P 073fe93a027145bfaaa6d07df7ad638f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfdc99c22b6c4564ab8571edfc3291ee in term 1.
I20260809 12:56:48.087898 28176 raft_consensus.cc:2454] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfdc99c22b6c4564ab8571edfc3291ee in term 1.
I20260809 12:56:48.088205 27955 leader_election.cc:302] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073fe93a027145bfaaa6d07df7ad638f, bfdc99c22b6c4564ab8571edfc3291ee; no voters:
I20260809 12:56:48.088551 28252 raft_consensus.cc:2781] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:48.089002 28252 raft_consensus.cc:686] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [term 1 LEADER]: Becoming Leader. State: Replica: bfdc99c22b6c4564ab8571edfc3291ee, State: Running, Role: LEADER
I20260809 12:56:48.089510 28252 consensus_queue.cc:237] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073fe93a027145bfaaa6d07df7ad638f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46451 } } peers { permanent_uuid: "bfdc99c22b6c4564ab8571edfc3291ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39139 } } peers { permanent_uuid: "a12d06067dd14c38aa086dc0ce3312df" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44381 } }
I20260809 12:56:48.094053 27730 catalog_manager.cc:5495] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee reported cstate change: term changed from 0 to 1, leader changed from <none> to bfdc99c22b6c4564ab8571edfc3291ee (127.21.235.66). New cstate: current_term: 1 leader_uuid: "bfdc99c22b6c4564ab8571edfc3291ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073fe93a027145bfaaa6d07df7ad638f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfdc99c22b6c4564ab8571edfc3291ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a12d06067dd14c38aa086dc0ce3312df" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44381 } health_report { overall_health: UNKNOWN } } }
I20260809 12:56:48.107646 28238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:48.105198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50452 (local address 127.21.235.66:39139)
0809 12:56:48.105518 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:48.105524 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:48.105632 (+ 108us) server_negotiation.cc:407] Connection header received
0809 12:56:48.105894 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:48.105898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:48.105978 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:48.106089 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:48.107034 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:48.107040 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:48.107044 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:48.107067 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:48.107101 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:48.107112 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:48.107236 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:48.107325 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 12:56:48.107387 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184}
I20260809 12:56:48.115890 27885 raft_consensus.cc:1261] T 59929299a4e24dd5a2d9069cf514c355 P 073fe93a027145bfaaa6d07df7ad638f [term 1 FOLLOWER]: Refusing update from remote peer bfdc99c22b6c4564ab8571edfc3291ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:48.116081 28176 raft_consensus.cc:1261] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df [term 1 FOLLOWER]: Refusing update from remote peer bfdc99c22b6c4564ab8571edfc3291ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:56:48.117002 28255 consensus_queue.cc:1035] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "a12d06067dd14c38aa086dc0ce3312df" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:56:48.117347 28252 consensus_queue.cc:1035] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "073fe93a027145bfaaa6d07df7ad638f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:56:48.129117 28258 mvcc.cc:205] Tried to move back new op lower bound from 7316603732432494592 to 7316603732338741248. Current Snapshot: MvccSnapshot[applied={T|T < 7316603732432494592}]
I20260809 12:56:48.135936 28260 mvcc.cc:205] Tried to move back new op lower bound from 7316603732432494592 to 7316603732338741248. Current Snapshot: MvccSnapshot[applied={T|T < 7316603732432494592}]
I20260809 12:56:48.145006 27730 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:56:48.145308 27730 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:56:48.149119 27730 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1ee1f71e7f244af92bbc341a6e3bf3c: Sending DeleteTablet for 3 replicas of tablet 59929299a4e24dd5a2d9069cf514c355
I20260809 12:56:48.150309 28007 tablet_service.cc:1514] Processing DeleteTablet for tablet 59929299a4e24dd5a2d9069cf514c355 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:48 UTC) from {username='slave'} at 127.0.0.1:50450
I20260809 12:56:48.150302 28153 tablet_service.cc:1514] Processing DeleteTablet for tablet 59929299a4e24dd5a2d9069cf514c355 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:48 UTC) from {username='slave'} at 127.0.0.1:50892
I20260809 12:56:48.150372 27864 tablet_service.cc:1514] Processing DeleteTablet for tablet 59929299a4e24dd5a2d9069cf514c355 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:56:48 UTC) from {username='slave'} at 127.0.0.1:33658
I20260809 12:56:48.152853 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 27799
I20260809 12:56:48.165443 28239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:48.154390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33672 (local address 127.21.235.65:46451)
0809 12:56:48.154613 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:48.154619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:48.160769 (+ 6150us) server_negotiation.cc:407] Connection header received
0809 12:56:48.161834 (+ 1065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:48.161844 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:48.162010 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:48.162107 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:48.164163 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:48.164173 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:48.164181 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:48.164208 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:48.164257 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:48.164280 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:48.164410 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:48.164899 (+ 489us) server_negotiation.cc:299] Negotiation successful
0809 12:56:48.164995 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:56:48.166785 27928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:48.169121 28267 tablet_replica.cc:330] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df: stopping tablet replica
I20260809 12:56:48.169761 28267 raft_consensus.cc:2229] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:56:48.171026 28268 tablet_replica.cc:330] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee: stopping tablet replica
I20260809 12:56:48.171758 28268 raft_consensus.cc:2229] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:56:48.172762 28268 raft_consensus.cc:2258] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:48.173013 28267 raft_consensus.cc:2258] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:56:48.176097 28268 ts_tablet_manager.cc:1905] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:48.177827 28267 ts_tablet_manager.cc:1905] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:56:48.188565 28267 ts_tablet_manager.cc:1918] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:56:48.188791 28267 log.cc:1196] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/59929299a4e24dd5a2d9069cf514c355
I20260809 12:56:48.189427 28267 ts_tablet_manager.cc:1939] T 59929299a4e24dd5a2d9069cf514c355 P a12d06067dd14c38aa086dc0ce3312df: Deleting consensus metadata
I20260809 12:56:48.189599 28268 ts_tablet_manager.cc:1918] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:56:48.189930 28268 log.cc:1196] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/59929299a4e24dd5a2d9069cf514c355
I20260809 12:56:48.190636 28268 ts_tablet_manager.cc:1939] T 59929299a4e24dd5a2d9069cf514c355 P bfdc99c22b6c4564ab8571edfc3291ee: Deleting consensus metadata
I20260809 12:56:48.191655 27714 catalog_manager.cc:4841] TS a12d06067dd14c38aa086dc0ce3312df (127.21.235.67:44381): tablet 59929299a4e24dd5a2d9069cf514c355 (table test-table [id=aa58347e5ed549d0936f4cab07ec8def]) successfully deleted
I20260809 12:56:48.192579 27715 catalog_manager.cc:4841] TS bfdc99c22b6c4564ab8571edfc3291ee (127.21.235.66:39139): tablet 59929299a4e24dd5a2d9069cf514c355 (table test-table [id=aa58347e5ed549d0936f4cab07ec8def]) successfully deleted
I20260809 12:56:48.328176 28269 tablet_replica.cc:330] T 59929299a4e24dd5a2d9069cf514c355 P 073fe93a027145bfaaa6d07df7ad638f: stopping tablet replica
I20260809 12:56:48.328727 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 27799
W20260809 12:56:48.354727 27714 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.235.65:46451 (error 108)
I20260809 12:56:48.355084 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 27938
I20260809 12:56:48.357322 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:48.356706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:46451 (local address 127.0.0.1:33678)
0809 12:56:48.356960 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:48.357125 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:46451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
W20260809 12:56:48.357610 27714 catalog_manager.cc:4584] TS 073fe93a027145bfaaa6d07df7ad638f (127.21.235.65:46451): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 59929299a4e24dd5a2d9069cf514c355: Network error: Client connection negotiation failed: client connection to 127.21.235.65:46451: connect: Connection refused (error 111)
I20260809 12:56:48.358680 28238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:48.356232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50462 (local address 127.21.235.66:39139)
0809 12:56:48.356412 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:48.356419 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:48.356481 (+ 62us) server_negotiation.cc:407] Connection header received
0809 12:56:48.356558 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:48.356566 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:48.356653 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:48.356752 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:48.357938 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:48.357945 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:48.357951 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:48.357972 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:48.358022 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:48.358039 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:48.358164 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:48.358293 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 12:56:48.358355 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 12:56:48.359476 28075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:48.411458 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:48.410953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:46451 (local address 127.0.0.1:33682)
0809 12:56:48.411206 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:48.411306 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:46451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260809 12:56:48.412513 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:48.412177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:46451 (local address 127.0.0.1:33688)
0809 12:56:48.412304 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:48.412383 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:46451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 12:56:48.470328 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 27938
I20260809 12:56:48.472764 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:48.472306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:46451 (local address 127.0.0.1:33696)
0809 12:56:48.472511 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:48.472614 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:46451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 12:56:48.473894 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:48.473536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:46451 (local address 127.0.0.1:33698)
0809 12:56:48.473679 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:48.473762 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:46451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 12:56:48.493221 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 28083
I20260809 12:56:48.496276 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:48.493993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50894 (local address 127.21.235.67:44381)
0809 12:56:48.494184 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:48.494191 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:48.494251 (+ 60us) server_negotiation.cc:407] Connection header received
0809 12:56:48.494561 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:48.494566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:48.494646 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:48.494757 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:48.495603 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:48.495609 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:48.495613 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:48.495639 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:48.495673 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:48.495686 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:48.495822 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:48.495917 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 12:56:48.495982 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 12:56:48.496901 28218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:48.562848 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:48.562251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:46451 (local address 127.0.0.1:33700)
0809 12:56:48.562526 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:48.562647 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:46451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 12:56:48.564023 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:48.563568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:46451 (local address 127.0.0.1:33704)
0809 12:56:48.563789 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:48.563873 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:46451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260809 12:56:48.585525 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 28083
I20260809 12:56:48.605592 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 27698
I20260809 12:56:48.606854 27767 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:48.714211 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 27698
2026-08-09T12:56:48Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2742 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 12:56:48.759086 22445 mini_oidc.cc:152] Starting JWKS server
I20260809 12:56:48.762075 22445 webserver.cc:460] Webserver started at https://127.0.0.1:36817/ using document root <none> and password file <none>
I20260809 12:56:48.762383 22445 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 12:56:48.763130 22445 webserver.cc:460] Webserver started at http://127.0.0.1:44237/ using document root <none> and password file <none>
2026-08-09T12:56:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:56:48Z Disabled control of system clock
I20260809 12:56:48.795827 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:42695
--webserver_interface=127.21.235.126
--webserver_port=0
--builtin_ntp_servers=127.21.235.84:34973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:42695
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36817/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/testchainca.cert with env {}
W20260809 12:56:49.133800 28284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:49.134184 28284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:49.134375 28284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:49.141853 28284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:56:49.142016 28284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:49.142112 28284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:49.142232 28284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:56:49.142378 28284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 12:56:49.142517 28284 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 12:56:49.142599 28284 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36817/jwks.json
I20260809 12:56:49.152719 28284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:34973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:42695
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:42695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36817/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:49.154137 28284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:49.155519 28284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:49.168886 28293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:49.168951 28291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:49.168977 28290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:49.169584 28284 server_base.cc:1034] running on GCE node
I20260809 12:56:49.170567 28284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:49.172154 28284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:49.173466 28284 hybrid_clock.cc:648] HybridClock initialized: now 1786280209173427 us; error 92 us; skew 500 ppm
I20260809 12:56:49.173861 28284 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:56:49.176640 28284 webserver.cc:460] Webserver started at http://127.21.235.126:45917/ using document root <none> and password file <none>
I20260809 12:56:49.177402 28284 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:49.177552 28284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:49.177901 28284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:49.181252 28284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "32ffe87d4fbb4e13a618ffed29ea1638"
format_stamp: "Formatted at 2026-08-09 12:56:49 on dist-test-slave-gp6n"
server_key: "b1d1442fdf2e2a40d86dfdedc5dec609"
server_key_iv: "619edc9da682cb5e0525bbbd13b660b8"
server_key_version: "encryptionkey@0"
I20260809 12:56:49.182183 28284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "32ffe87d4fbb4e13a618ffed29ea1638"
format_stamp: "Formatted at 2026-08-09 12:56:49 on dist-test-slave-gp6n"
server_key: "b1d1442fdf2e2a40d86dfdedc5dec609"
server_key_iv: "619edc9da682cb5e0525bbbd13b660b8"
server_key_version: "encryptionkey@0"
I20260809 12:56:49.187945 28284 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 12:56:49.191712 28300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:49.193248 28284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:56:49.193464 28284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "32ffe87d4fbb4e13a618ffed29ea1638"
format_stamp: "Formatted at 2026-08-09 12:56:49 on dist-test-slave-gp6n"
server_key: "b1d1442fdf2e2a40d86dfdedc5dec609"
server_key_iv: "619edc9da682cb5e0525bbbd13b660b8"
server_key_version: "encryptionkey@0"
I20260809 12:56:49.193707 28284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:49.214239 28284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:49.215220 28284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:49.215592 28284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:49.236603 28284 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:42695
I20260809 12:56:49.236667 28358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:42695 every 8 connection(s)
I20260809 12:56:49.238153 28284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:56:49.242390 28359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:49.247567 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 28284
I20260809 12:56:49.247830 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:56:49.248764 22445 external_mini_cluster.cc:1428] Setting key 9bfb6e05f504006af247d7c7eff4ec23
I20260809 12:56:49.255280 28359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638: Bootstrap starting.
I20260809 12:56:49.258760 28359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:49.260025 28359 log.cc:824] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:49.263203 28362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:49.250997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35990 (local address 127.21.235.126:42695)
0809 12:56:49.252052 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:49.252063 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:49.252103 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:56:49.253893 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:49.253916 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:49.254361 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:49.254811 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:49.255404 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:49.256779 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:49.258170 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:49.258966 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:49.259178 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:49.260452 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:49.260460 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:49.260473 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:49.261317 (+ 844us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:49.261419 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:49.261453 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:49.261593 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:49.261816 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 12:56:49.262136 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":403,"threads_started":1}
I20260809 12:56:49.263429 28359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638: No bootstrap required, opened a new log
I20260809 12:56:49.267244 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:56:49.272893 28359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ffe87d4fbb4e13a618ffed29ea1638" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42695 } }
I20260809 12:56:49.273401 28359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:49.273535 28359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ffe87d4fbb4e13a618ffed29ea1638, State: Initialized, Role: FOLLOWER
I20260809 12:56:49.274257 28359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ffe87d4fbb4e13a618ffed29ea1638" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42695 } }
I20260809 12:56:49.274628 28359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:56:49.274798 28359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:56:49.275009 28359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:49.277647 28359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ffe87d4fbb4e13a618ffed29ea1638" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42695 } }
I20260809 12:56:49.278194 28359 leader_election.cc:302] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32ffe87d4fbb4e13a618ffed29ea1638; no voters:
I20260809 12:56:49.278942 28359 leader_election.cc:288] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:56:49.279047 28365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:49.279883 28365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638 [term 1 LEADER]: Becoming Leader. State: Replica: 32ffe87d4fbb4e13a618ffed29ea1638, State: Running, Role: LEADER
I20260809 12:56:49.280439 28365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ffe87d4fbb4e13a618ffed29ea1638" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42695 } }
I20260809 12:56:49.281278 28359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:56:49.283324 28367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32ffe87d4fbb4e13a618ffed29ea1638" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ffe87d4fbb4e13a618ffed29ea1638" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42695 } } }
I20260809 12:56:49.283844 28367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638 [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:49.283810 28366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32ffe87d4fbb4e13a618ffed29ea1638. Latest consensus state: current_term: 1 leader_uuid: "32ffe87d4fbb4e13a618ffed29ea1638" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ffe87d4fbb4e13a618ffed29ea1638" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42695 } } }
I20260809 12:56:49.284106 28366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638 [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:49.284839 28375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:56:49.290661 28375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:56:49.299753 28375 catalog_manager.cc:1347] Generated new cluster ID: 1052033b62154183b6784b8d41261cd2
I20260809 12:56:49.299891 28375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:56:49.313011 28375 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:56:49.314514 28375 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:56:49.322446 28375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32ffe87d4fbb4e13a618ffed29ea1638: Generated new TSK 0
I20260809 12:56:49.323380 28375 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 12:56:49.372645 28393 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 12:56:49.372947 28393 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 12:56:49.374181 28362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:49.348850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35992 (local address 127.21.235.126:42695)
0809 12:56:49.349065 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:49.349071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:49.349906 (+ 835us) server_negotiation.cc:407] Connection header received
0809 12:56:49.350267 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:49.350274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:49.350426 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:49.350586 (+ 160us) server_negotiation.cc:226] Negotiated authn=JWT
0809 12:56:49.351213 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:49.352372 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:49.353997 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:49.354732 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:49.354863 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:49.355474 (+ 611us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 12:56:49.373459 (+ 17985us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36817/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 12:56:49.373866 (+ 407us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35992: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36817/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 12:56:49.374325 28362 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35992: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36817/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 12:56:49.376096 28392 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:56:49.348841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42695 (local address 127.0.0.1:35992)
0809 12:56:49.349772 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:49.349806 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:49.349984 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:49.350658 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:49.350662 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:49.350673 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0809 12:56:49.351051 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:49.351060 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:49.352548 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:49.352553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:49.353844 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:49.353857 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:49.354903 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:49.354908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:49.355057 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:49.355128 (+ 71us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 12:56:49.373817 (+ 18689us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36817/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 12:56:49.374157 (+ 340us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.235.126:42695: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36817/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":475,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.21.235.126:42695: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36817/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 12:56:49.378187 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 28284
I20260809 12:56:49.379498 28354 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:49.478189 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 28284
2026-08-09T12:56:49Z chronyd exiting
I20260809 12:56:49.530381 22445 test_util.cc:175] -----------------------------------------------
I20260809 12:56:49.530619 22445 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280163688078-22445-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (776 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 12:56:49.534687 22445 mini_oidc.cc:152] Starting JWKS server
I20260809 12:56:49.536518 22445 webserver.cc:460] Webserver started at https://127.0.0.1:40491/ using document root <none> and password file <none>
I20260809 12:56:49.536764 22445 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 12:56:49.537456 22445 webserver.cc:460] Webserver started at http://127.0.0.1:45465/ using document root <none> and password file <none>
2026-08-09T12:56:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:56:49Z Disabled control of system clock
I20260809 12:56:49.569928 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:42509
--webserver_interface=127.21.235.126
--webserver_port=0
--builtin_ntp_servers=127.21.235.84:45395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:42509
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40491/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/testchainca.cert with env {}
W20260809 12:56:49.898270 28400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:49.898698 28400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:49.898865 28400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:49.905948 28400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:56:49.906095 28400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:49.906191 28400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:49.906281 28400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:56:49.906369 28400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 12:56:49.906484 28400 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 12:56:49.906569 28400 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40491/jwks.json
I20260809 12:56:49.916500 28400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:45395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:42509
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:42509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40491/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:49.917884 28400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:49.919247 28400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:49.931740 28406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:49.931687 28410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:49.931792 28407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:49.932716 28400 server_base.cc:1034] running on GCE node
I20260809 12:56:49.933488 28400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:49.935075 28400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:49.936311 28400 hybrid_clock.cc:648] HybridClock initialized: now 1786280209936252 us; error 28 us; skew 500 ppm
I20260809 12:56:49.936748 28400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:56:49.943624 28400 webserver.cc:460] Webserver started at http://127.21.235.126:36107/ using document root <none> and password file <none>
I20260809 12:56:49.944387 28400 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:49.944516 28400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:49.944873 28400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:49.947885 28400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "5e6e75639de74f61a979e3d2d1f44b1f"
format_stamp: "Formatted at 2026-08-09 12:56:49 on dist-test-slave-gp6n"
server_key: "efcf964d3b6140eae0188a2e22956fa0"
server_key_iv: "31b2f2da33fb3aa7376eba0aed2f7526"
server_key_version: "encryptionkey@0"
I20260809 12:56:49.948765 28400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "5e6e75639de74f61a979e3d2d1f44b1f"
format_stamp: "Formatted at 2026-08-09 12:56:49 on dist-test-slave-gp6n"
server_key: "efcf964d3b6140eae0188a2e22956fa0"
server_key_iv: "31b2f2da33fb3aa7376eba0aed2f7526"
server_key_version: "encryptionkey@0"
I20260809 12:56:49.954083 28400 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:56:49.957573 28417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:49.959088 28400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:56:49.959272 28400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "5e6e75639de74f61a979e3d2d1f44b1f"
format_stamp: "Formatted at 2026-08-09 12:56:49 on dist-test-slave-gp6n"
server_key: "efcf964d3b6140eae0188a2e22956fa0"
server_key_iv: "31b2f2da33fb3aa7376eba0aed2f7526"
server_key_version: "encryptionkey@0"
I20260809 12:56:49.959498 28400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:49.992007 28400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:49.992996 28400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:49.993361 28400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:50.014050 28400 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:42509
I20260809 12:56:50.014072 28471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:42509 every 8 connection(s)
I20260809 12:56:50.015612 28400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:56:50.019510 28472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:50.020540 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 28400
I20260809 12:56:50.020751 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:56:50.021634 22445 external_mini_cluster.cc:1428] Setting key c5e5bc67114b6ac0ca32a00408bf458a
I20260809 12:56:50.034266 28472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f: Bootstrap starting.
I20260809 12:56:50.036034 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:50.023719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59994 (local address 127.21.235.126:42509)
0809 12:56:50.024841 (+ 1122us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:50.024856 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:50.024899 (+ 43us) server_negotiation.cc:407] Connection header received
0809 12:56:50.026515 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:50.026538 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:50.027035 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:50.027384 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:50.027999 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:50.029342 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:50.031256 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:50.032071 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:50.032215 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:50.033499 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:50.033507 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:50.033521 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:50.034107 (+ 586us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:50.034220 (+ 113us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:50.034251 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:50.034351 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:50.034657 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 12:56:50.035024 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":414,"threads_started":1}
I20260809 12:56:50.037932 28472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:50.039250 28472 log.cc:824] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:50.040409 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:56:50.043020 28472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f: No bootstrap required, opened a new log
I20260809 12:56:50.050567 28472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6e75639de74f61a979e3d2d1f44b1f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42509 } }
I20260809 12:56:50.050987 28472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:50.051093 28472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6e75639de74f61a979e3d2d1f44b1f, State: Initialized, Role: FOLLOWER
I20260809 12:56:50.051661 28472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6e75639de74f61a979e3d2d1f44b1f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42509 } }
I20260809 12:56:50.051905 28472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:56:50.052058 28472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:56:50.052237 28472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:50.054680 28472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6e75639de74f61a979e3d2d1f44b1f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42509 } }
I20260809 12:56:50.055197 28472 leader_election.cc:302] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e6e75639de74f61a979e3d2d1f44b1f; no voters:
I20260809 12:56:50.055794 28472 leader_election.cc:288] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:56:50.055968 28478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:50.056778 28478 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f [term 1 LEADER]: Becoming Leader. State: Replica: 5e6e75639de74f61a979e3d2d1f44b1f, State: Running, Role: LEADER
I20260809 12:56:50.057348 28478 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6e75639de74f61a979e3d2d1f44b1f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42509 } }
I20260809 12:56:50.058064 28472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:56:50.060947 28479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e6e75639de74f61a979e3d2d1f44b1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6e75639de74f61a979e3d2d1f44b1f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42509 } } }
I20260809 12:56:50.060967 28480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e6e75639de74f61a979e3d2d1f44b1f. Latest consensus state: current_term: 1 leader_uuid: "5e6e75639de74f61a979e3d2d1f44b1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e6e75639de74f61a979e3d2d1f44b1f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42509 } } }
I20260809 12:56:50.061354 28479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:50.061430 28480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:50.071305 28487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:56:50.075668 28487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:56:50.083732 28487 catalog_manager.cc:1347] Generated new cluster ID: 3b93d967aba446529c2b6a64ea5d280f
I20260809 12:56:50.083860 28487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:56:50.100481 28487 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:56:50.101823 28487 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:56:50.110200 28487 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e6e75639de74f61a979e3d2d1f44b1f: Generated new TSK 0
I20260809 12:56:50.111048 28487 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 12:56:50.165563 28504 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 12:56:50.165711 28504 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 12:56:50.166778 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:50.134623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60000 (local address 127.21.235.126:42509)
0809 12:56:50.134866 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:50.134880 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:50.135443 (+ 563us) server_negotiation.cc:407] Connection header received
0809 12:56:50.135691 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:50.135695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:50.135766 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:50.135909 (+ 143us) server_negotiation.cc:226] Negotiated authn=JWT
0809 12:56:50.136547 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:50.137407 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:50.139786 (+ 2379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:50.140549 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:50.140657 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:50.141370 (+ 713us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 12:56:50.166115 (+ 24745us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40491/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 12:56:50.166538 (+ 423us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60000: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40491/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 12:56:50.166930 28475 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60000: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40491/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 12:56:50.167255 28503 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:56:50.134595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42509 (local address 127.0.0.1:60000)
0809 12:56:50.135281 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:50.135305 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:50.135472 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:50.135970 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:50.135976 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:50.135994 (+ 18us) client_negotiation.cc:189] Negotiated authn=JWT
0809 12:56:50.136351 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:50.136369 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:50.137564 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:50.137598 (+ 34us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:50.139590 (+ 1992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:50.139606 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:50.140754 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:50.140770 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:50.141003 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:50.141060 (+ 57us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 12:56:50.166432 (+ 25372us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40491/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 12:56:50.166772 (+ 340us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.235.126:42509: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40491/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":367,"threads_started":1}
I20260809 12:56:50.168511 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 28400
I20260809 12:56:50.169914 28467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:50.320335 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 28400
2026-08-09T12:56:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (864 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 12:56:50.402884 22445 mini_oidc.cc:152] Starting JWKS server
I20260809 12:56:50.406642 22445 webserver.cc:460] Webserver started at https://127.0.0.1:38029/ using document root <none> and password file <none>
I20260809 12:56:50.407183 22445 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 12:56:50.408480 22445 webserver.cc:460] Webserver started at http://127.0.0.1:38963/ using document root <none> and password file <none>
2026-08-09T12:56:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:56:50Z Disabled control of system clock
I20260809 12:56:50.466717 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:46437
--webserver_interface=127.21.235.126
--webserver_port=0
--builtin_ntp_servers=127.21.235.84:34283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:46437
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38029/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 12:56:51.145445 28513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:51.146175 28513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:51.146493 28513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:51.162617 28513 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:56:51.162891 28513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:51.163144 28513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:51.163331 28513 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:56:51.163477 28513 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 12:56:51.163659 28513 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 12:56:51.163802 28513 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38029/jwks.json
I20260809 12:56:51.188541 28513 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:34283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:46437
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:46437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38029/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:51.192346 28513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:51.197395 28513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:51.224915 28519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:51.225565 28518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:51.226280 28521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:51.227213 28513 server_base.cc:1034] running on GCE node
I20260809 12:56:51.228564 28513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:51.231375 28513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:51.234902 28513 hybrid_clock.cc:648] HybridClock initialized: now 1786280211234750 us; error 154 us; skew 500 ppm
I20260809 12:56:51.235751 28513 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:56:51.240711 28513 webserver.cc:460] Webserver started at http://127.21.235.126:38259/ using document root <none> and password file <none>
I20260809 12:56:51.241971 28513 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:51.242219 28513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:51.242709 28513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:51.248198 28513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "c839557d65b5427a9feb64f79c99d58f"
format_stamp: "Formatted at 2026-08-09 12:56:51 on dist-test-slave-gp6n"
server_key: "a6b6891c6ab39bbb807f5519ed1d1d23"
server_key_iv: "4b08c3cfa82826bbb754fb099a341ccc"
server_key_version: "encryptionkey@0"
I20260809 12:56:51.249562 28513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "c839557d65b5427a9feb64f79c99d58f"
format_stamp: "Formatted at 2026-08-09 12:56:51 on dist-test-slave-gp6n"
server_key: "a6b6891c6ab39bbb807f5519ed1d1d23"
server_key_iv: "4b08c3cfa82826bbb754fb099a341ccc"
server_key_version: "encryptionkey@0"
I20260809 12:56:51.258931 28513 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.010s sys 0.001s
I20260809 12:56:51.264864 28527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:51.267336 28513 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.003s
I20260809 12:56:51.267545 28513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "c839557d65b5427a9feb64f79c99d58f"
format_stamp: "Formatted at 2026-08-09 12:56:51 on dist-test-slave-gp6n"
server_key: "a6b6891c6ab39bbb807f5519ed1d1d23"
server_key_iv: "4b08c3cfa82826bbb754fb099a341ccc"
server_key_version: "encryptionkey@0"
I20260809 12:56:51.267865 28513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:51.295966 28513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:51.297341 28513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:51.297747 28513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:51.330205 28513 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:46437
I20260809 12:56:51.330230 28586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:46437 every 8 connection(s)
I20260809 12:56:51.332571 28513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:56:51.337049 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 28513
I20260809 12:56:51.337466 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:56:51.339275 22445 external_mini_cluster.cc:1428] Setting key 8c9ca3364099b191aa557f33c7373709
I20260809 12:56:51.340721 28587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:51.367172 28590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:51.342789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39648 (local address 127.21.235.126:46437)
0809 12:56:51.344685 (+ 1896us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:51.344717 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:51.344822 (+ 105us) server_negotiation.cc:407] Connection header received
0809 12:56:51.347591 (+ 2769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:51.347642 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:51.348980 (+ 1338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:51.349796 (+ 816us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:51.350895 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:51.354764 (+ 3869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:51.356976 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:51.359112 (+ 2136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:51.359481 (+ 369us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:51.361280 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:51.361301 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:51.361342 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:51.362880 (+ 1538us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:51.363068 (+ 188us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:51.363131 (+ 63us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:51.363349 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:51.363731 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 12:56:51.364412 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":904,"thread_start_us":540,"threads_started":1}
I20260809 12:56:51.369426 28587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f: Bootstrap starting.
I20260809 12:56:51.374186 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:56:51.375558 28587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:51.377817 28587 log.cc:824] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:51.383353 28587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f: No bootstrap required, opened a new log
I20260809 12:56:51.397289 28587 raft_consensus.cc:348] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c839557d65b5427a9feb64f79c99d58f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46437 } }
I20260809 12:56:51.397908 28587 raft_consensus.cc:374] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:51.398110 28587 raft_consensus.cc:729] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c839557d65b5427a9feb64f79c99d58f, State: Initialized, Role: FOLLOWER
I20260809 12:56:51.399333 28587 consensus_queue.cc:260] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c839557d65b5427a9feb64f79c99d58f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46437 } }
I20260809 12:56:51.399950 28587 raft_consensus.cc:388] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:56:51.400101 28587 raft_consensus.cc:482] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:56:51.400305 28587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:51.403970 28587 raft_consensus.cc:504] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c839557d65b5427a9feb64f79c99d58f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46437 } }
I20260809 12:56:51.404716 28587 leader_election.cc:302] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c839557d65b5427a9feb64f79c99d58f; no voters:
I20260809 12:56:51.405604 28587 leader_election.cc:288] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:56:51.405879 28592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:51.408226 28592 raft_consensus.cc:686] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f [term 1 LEADER]: Becoming Leader. State: Replica: c839557d65b5427a9feb64f79c99d58f, State: Running, Role: LEADER
I20260809 12:56:51.409124 28592 consensus_queue.cc:237] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c839557d65b5427a9feb64f79c99d58f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46437 } }
I20260809 12:56:51.409794 28587 sys_catalog.cc:564] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:56:51.414851 28594 sys_catalog.cc:455] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c839557d65b5427a9feb64f79c99d58f. Latest consensus state: current_term: 1 leader_uuid: "c839557d65b5427a9feb64f79c99d58f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c839557d65b5427a9feb64f79c99d58f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46437 } } }
I20260809 12:56:51.415716 28594 sys_catalog.cc:458] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:51.416731 28593 sys_catalog.cc:455] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c839557d65b5427a9feb64f79c99d58f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c839557d65b5427a9feb64f79c99d58f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46437 } } }
I20260809 12:56:51.417434 28593 sys_catalog.cc:458] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:51.425899 28601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:56:51.431779 28601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:56:51.448179 28601 catalog_manager.cc:1347] Generated new cluster ID: d64632e4ecf94c8d919c8faaf0cc8c39
I20260809 12:56:51.448434 28601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:56:51.488250 28601 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:56:51.491879 28601 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:56:51.513036 28601 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c839557d65b5427a9feb64f79c99d58f: Generated new TSK 0
I20260809 12:56:51.514544 28601 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:56:51.564594 28590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:51.549644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45220 (local address 127.21.235.126:46437)
0809 12:56:51.550012 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:51.550041 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:51.550863 (+ 822us) server_negotiation.cc:407] Connection header received
0809 12:56:51.551325 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:51.551332 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:51.551532 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:51.551891 (+ 359us) server_negotiation.cc:226] Negotiated authn=JWT
0809 12:56:51.552846 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:51.554018 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:51.557657 (+ 3639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:51.558533 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:51.558729 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:51.559798 (+ 1069us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 12:56:51.563367 (+ 3569us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38029/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 12:56:51.563943 (+ 576us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45220: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38029/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":98}
W20260809 12:56:51.564842 28590 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45220: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38029/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 12:56:51.565153 28617 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:56:51.549543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:46437 (local address 127.0.0.1:45220)
0809 12:56:51.550679 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:51.550722 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:51.550957 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:51.551905 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:51.551912 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:51.551935 (+ 23us) client_negotiation.cc:189] Negotiated authn=JWT
0809 12:56:51.552604 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:51.552638 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:51.554226 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:51.554233 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:51.557380 (+ 3147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:51.557399 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:51.558864 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:51.558872 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:51.559304 (+ 432us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:51.559398 (+ 94us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 12:56:51.563701 (+ 4303us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38029/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 12:56:51.564208 (+ 507us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.235.126:46437: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38029/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":871,"thread_start_us":563,"threads_started":1}
I20260809 12:56:51.567196 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 28513
I20260809 12:56:51.569384 28581 generic_service.cc:196] Checking for leaks (request via RPC)
E20260809 12:56:51.591879 28618 webserver.cc:533] Webserver: sslize failed:
E20260809 12:56:51.592123 28618 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 12:56:51.705984 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 28513
2026-08-09T12:56:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (1378 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T12:56:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:56:51Z Disabled control of system clock
I20260809 12:56:51.839901 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:36041
--webserver_interface=127.21.235.126
--webserver_port=0
--builtin_ntp_servers=127.21.235.84:36759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:36041
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/testchainca.cert with env {}
W20260809 12:56:52.449473 28624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:52.450109 28624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:52.450335 28624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:52.462627 28624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:56:52.462827 28624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:52.462910 28624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:52.463009 28624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:56:52.463076 28624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 12:56:52.463183 28624 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 12:56:52.463253 28624 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 12:56:52.480345 28624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:36759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:36041
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:36041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:52.482838 28624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:52.485117 28624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:52.506224 28631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:52.506278 28633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:52.506124 28630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:52.512637 28624 server_base.cc:1034] running on GCE node
I20260809 12:56:52.514295 28624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:52.516917 28624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:52.518198 28624 hybrid_clock.cc:648] HybridClock initialized: now 1786280212518126 us; error 63 us; skew 500 ppm
I20260809 12:56:52.518843 28624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:56:52.522771 28624 webserver.cc:460] Webserver started at http://127.21.235.126:35027/ using document root <none> and password file <none>
I20260809 12:56:52.524003 28624 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:52.524226 28624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:52.524753 28624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:52.530368 28624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "6341d047a3764242b05f945a527dbf9d"
format_stamp: "Formatted at 2026-08-09 12:56:52 on dist-test-slave-gp6n"
server_key: "8db0934ba840a98f8763ec9b38150283"
server_key_iv: "d3becbd1eda26e05b7fa0098cc33f4c7"
server_key_version: "encryptionkey@0"
I20260809 12:56:52.532066 28624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "6341d047a3764242b05f945a527dbf9d"
format_stamp: "Formatted at 2026-08-09 12:56:52 on dist-test-slave-gp6n"
server_key: "8db0934ba840a98f8763ec9b38150283"
server_key_iv: "d3becbd1eda26e05b7fa0098cc33f4c7"
server_key_version: "encryptionkey@0"
I20260809 12:56:52.541415 28624 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.009s sys 0.000s
I20260809 12:56:52.547698 28639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:52.550345 28624 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:56:52.550730 28624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "6341d047a3764242b05f945a527dbf9d"
format_stamp: "Formatted at 2026-08-09 12:56:52 on dist-test-slave-gp6n"
server_key: "8db0934ba840a98f8763ec9b38150283"
server_key_iv: "d3becbd1eda26e05b7fa0098cc33f4c7"
server_key_version: "encryptionkey@0"
I20260809 12:56:52.551126 28624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:52.570619 28624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:52.572046 28624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:52.572511 28624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:52.604040 28624 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:36041
I20260809 12:56:52.604067 28695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:36041 every 8 connection(s)
I20260809 12:56:52.606580 28624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:56:52.613343 28696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:52.614642 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 28624
I20260809 12:56:52.614970 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:56:52.616153 22445 external_mini_cluster.cc:1428] Setting key a79ab961826a83a5ad49c6b1123f28a9
I20260809 12:56:52.635414 28696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d: Bootstrap starting.
I20260809 12:56:52.640372 28699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:52.619774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42966 (local address 127.21.235.126:36041)
0809 12:56:52.621679 (+ 1905us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:52.621704 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:52.621766 (+ 62us) server_negotiation.cc:407] Connection header received
0809 12:56:52.624739 (+ 2973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:52.624783 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:52.625690 (+ 907us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:52.626506 (+ 816us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:52.627704 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:52.629770 (+ 2066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:52.632030 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:52.633274 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:52.633663 (+ 389us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:52.635917 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:52.635938 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:52.635963 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:56:52.636877 (+ 914us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:52.637029 (+ 152us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:56:52.637063 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:52.637323 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:52.637742 (+ 419us) server_negotiation.cc:299] Negotiation successful
0809 12:56:52.638345 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1148,"thread_start_us":735,"threads_started":1}
I20260809 12:56:52.642009 28696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:52.643393 28696 log.cc:824] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:52.645778 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:56:52.646695 28696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d: No bootstrap required, opened a new log
I20260809 12:56:52.654757 28696 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6341d047a3764242b05f945a527dbf9d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36041 } }
I20260809 12:56:52.655198 28696 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:52.655318 28696 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6341d047a3764242b05f945a527dbf9d, State: Initialized, Role: FOLLOWER
I20260809 12:56:52.655898 28696 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6341d047a3764242b05f945a527dbf9d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36041 } }
I20260809 12:56:52.656170 28696 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:56:52.656320 28696 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:56:52.656505 28696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:52.659209 28696 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6341d047a3764242b05f945a527dbf9d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36041 } }
I20260809 12:56:52.659721 28696 leader_election.cc:302] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6341d047a3764242b05f945a527dbf9d; no voters:
I20260809 12:56:52.660328 28696 leader_election.cc:288] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:56:52.660502 28702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:52.661330 28702 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d [term 1 LEADER]: Becoming Leader. State: Replica: 6341d047a3764242b05f945a527dbf9d, State: Running, Role: LEADER
I20260809 12:56:52.661963 28702 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6341d047a3764242b05f945a527dbf9d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36041 } }
I20260809 12:56:52.662662 28696 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:56:52.665467 28704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6341d047a3764242b05f945a527dbf9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6341d047a3764242b05f945a527dbf9d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36041 } } }
I20260809 12:56:52.666577 28704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:52.665485 28705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6341d047a3764242b05f945a527dbf9d. Latest consensus state: current_term: 1 leader_uuid: "6341d047a3764242b05f945a527dbf9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6341d047a3764242b05f945a527dbf9d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36041 } } }
I20260809 12:56:52.666728 28705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:52.670392 28711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:56:52.676054 28711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:56:52.685297 28711 catalog_manager.cc:1347] Generated new cluster ID: 444a64e8d68d4c13b07585ef8d39e1d7
I20260809 12:56:52.685439 28711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:56:52.700098 28711 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:56:52.701522 28711 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:56:52.714216 28711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6341d047a3764242b05f945a527dbf9d: Generated new TSK 0
I20260809 12:56:52.715262 28711 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:56:52.801535 28699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:52.733086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42970 (local address 127.21.235.126:36041)
0809 12:56:52.733291 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:52.733297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:52.733715 (+ 418us) server_negotiation.cc:407] Connection header received
0809 12:56:52.733935 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:52.733942 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:52.734058 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:52.734249 (+ 191us) server_negotiation.cc:226] Negotiated authn=JWT
0809 12:56:52.734818 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:52.735955 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:52.737463 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:52.738180 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:52.738298 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:52.738867 (+ 569us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 12:56:52.800730 (+ 61863us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 12:56:52.801191 (+ 461us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42970: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":77}
W20260809 12:56:52.801638 28730 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:56:52.732920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:36041 (local address 127.0.0.1:42970)
0809 12:56:52.733604 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:52.733622 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:52.733751 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:52.734303 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:52.734307 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:52.734337 (+ 30us) client_negotiation.cc:189] Negotiated authn=JWT
0809 12:56:52.734654 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:52.734663 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:52.736142 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:52.736147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:52.737307 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:52.737318 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:52.738349 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:52.738355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:52.738552 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:52.738587 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 12:56:52.801053 (+ 62466us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 12:56:52.801315 (+ 262us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.235.126:36041: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":354,"threads_started":1}
W20260809 12:56:52.801688 28699 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42970: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260809 12:56:52.802873 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 28624
I20260809 12:56:52.804430 28689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:52.906148 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 28624
2026-08-09T12:56:52Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1166 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (437 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (421 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:56:53 dist-test-slave-gp6n krb5kdc[28745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:56:53 dist-test-slave-gp6n krb5kdc[28745](info): set up 2 sockets
Aug 09 12:56:53 dist-test-slave-gp6n krb5kdc[28745](info): commencing operation
krb5kdc: starting...
W20260809 12:56:55.019534 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:56:55 dist-test-slave-gp6n krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:56:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:56:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 12:56:55.266809 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:46281
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:42091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:46281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:55.614717 28759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:55.614989 28759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:55.615163 28759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:55.622196 28759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:56:55.622327 28759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:55.622463 28759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:55.622570 28759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:56:55.622620 28759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:56:55.632483 28759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:42091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:46281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:46281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:55.633843 28759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:55.635178 28759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:55.646869 28764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:55.646952 28765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:55.646914 28767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:55.648017 28759 server_base.cc:1034] running on GCE node
I20260809 12:56:55.648883 28759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:55.650153 28759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:55.651392 28759 hybrid_clock.cc:648] HybridClock initialized: now 1786280215651353 us; error 34 us; skew 500 ppm
I20260809 12:56:55.651683 28759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:55 dist-test-slave-gp6n krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280215, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:55.657660 28759 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:56:55.659206 28759 webserver.cc:460] Webserver started at http://127.21.235.126:41393/ using document root <none> and password file <none>
I20260809 12:56:55.659881 28759 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:55.660013 28759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:55.660348 28759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:55.663487 28759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "a7e2b8eca8f642f68a54cfd9c7c04dbb"
format_stamp: "Formatted at 2026-08-09 12:56:55 on dist-test-slave-gp6n"
server_key: "874feea7c319805de800919f097f048e"
server_key_iv: "4a1d0a825057179d763af48ae013a430"
server_key_version: "encryptionkey@0"
I20260809 12:56:55.664297 28759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "a7e2b8eca8f642f68a54cfd9c7c04dbb"
format_stamp: "Formatted at 2026-08-09 12:56:55 on dist-test-slave-gp6n"
server_key: "874feea7c319805de800919f097f048e"
server_key_iv: "4a1d0a825057179d763af48ae013a430"
server_key_version: "encryptionkey@0"
I20260809 12:56:55.669777 28759 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 12:56:55.673441 28777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:55.674896 28759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:56:55.675082 28759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "a7e2b8eca8f642f68a54cfd9c7c04dbb"
format_stamp: "Formatted at 2026-08-09 12:56:55 on dist-test-slave-gp6n"
server_key: "874feea7c319805de800919f097f048e"
server_key_iv: "4a1d0a825057179d763af48ae013a430"
server_key_version: "encryptionkey@0"
I20260809 12:56:55.675292 28759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:55.685782 28759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:55.690002 28759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:55.690305 28759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:55.708647 28759 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:46281
I20260809 12:56:55.708685 28835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:46281 every 8 connection(s)
I20260809 12:56:55.710100 28759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:56:55.713872 28836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:55.720362 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 28759
I20260809 12:56:55.720605 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:56:55.721426 22445 external_mini_cluster.cc:1428] Setting key ad65c48de933aa77c22abbb523552ea4
I20260809 12:56:55.725075 28836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb: Bootstrap starting.
I20260809 12:56:55.728688 28836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:55.729815 28836 log.cc:824] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb: Log is configured to *not* fsync() on all Append() calls
I20260809 12:56:55.732576 28836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb: No bootstrap required, opened a new log
Aug 09 12:56:55 dist-test-slave-gp6n krb5kdc[28745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:55.740453 28836 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e2b8eca8f642f68a54cfd9c7c04dbb" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46281 } }
I20260809 12:56:55.740851 28836 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:55.740972 28836 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e2b8eca8f642f68a54cfd9c7c04dbb, State: Initialized, Role: FOLLOWER
I20260809 12:56:55.741720 28836 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e2b8eca8f642f68a54cfd9c7c04dbb" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46281 } }
I20260809 12:56:55.742030 28836 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:56:55.742183 28836 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:56:55.742393 28836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:55.744546 28839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:55.723570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48918 (local address 127.21.235.126:46281)
0809 12:56:55.724370 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:55.724382 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:55.724423 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:56:55.726192 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:55.726215 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:55.726686 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:55.727024 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:55.728867 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:55.730288 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:55.731604 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:55.732266 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:55.732461 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:55.736457 (+ 3996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:55.736464 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:55.736481 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:55.736523 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:55.739763 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:55.740426 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:55.740433 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:55.740443 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:55.740615 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:55.741092 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:55.741097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:55.741101 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:55.741785 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:55.742105 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:55.742603 (+ 498us) server_negotiation.cc:299] Negotiation successful
0809 12:56:55.743063 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":398,"threads_started":1}
I20260809 12:56:55.745765 28836 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e2b8eca8f642f68a54cfd9c7c04dbb" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46281 } }
I20260809 12:56:55.746304 28836 leader_election.cc:302] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7e2b8eca8f642f68a54cfd9c7c04dbb; no voters:
I20260809 12:56:55.747016 28836 leader_election.cc:288] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:56:55.747206 28841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:55.747888 28841 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb [term 1 LEADER]: Becoming Leader. State: Replica: a7e2b8eca8f642f68a54cfd9c7c04dbb, State: Running, Role: LEADER
I20260809 12:56:55.748522 28841 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e2b8eca8f642f68a54cfd9c7c04dbb" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46281 } }
I20260809 12:56:55.749670 28836 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:56:55.751523 28843 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7e2b8eca8f642f68a54cfd9c7c04dbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e2b8eca8f642f68a54cfd9c7c04dbb" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46281 } } }
I20260809 12:56:55.751874 28843 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:55.752969 28842 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7e2b8eca8f642f68a54cfd9c7c04dbb. Latest consensus state: current_term: 1 leader_uuid: "a7e2b8eca8f642f68a54cfd9c7c04dbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e2b8eca8f642f68a54cfd9c7c04dbb" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46281 } } }
I20260809 12:56:55.753429 28842 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:55.754595 28847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:56:55.759871 28847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:56:55.769954 28847 catalog_manager.cc:1347] Generated new cluster ID: d3eb8161d71d453fa08e9bde18de22ad
I20260809 12:56:55.770080 28847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:56:55.799608 28847 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:56:55.801493 28847 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:56:55.809247 28847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7e2b8eca8f642f68a54cfd9c7c04dbb: Generated new TSK 0
I20260809 12:56:55.810292 28847 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
I20260809 12:56:55.871415 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:46281
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:42091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:56.216298 28864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:56.216717 28864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:56.216919 28864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:56.228345 28864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:56.228472 28864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:56.228590 28864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:56:56.239079 28864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:42091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:46281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:56.240375 28864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:56.241662 28864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:56.255317 28869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:56.255270 28870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:56.255241 28873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:56.259968 28864 server_base.cc:1034] running on GCE node
I20260809 12:56:56.260677 28864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:56.261642 28864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:56.262879 28864 hybrid_clock.cc:648] HybridClock initialized: now 1786280216262834 us; error 56 us; skew 500 ppm
I20260809 12:56:56.263226 28864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:56 dist-test-slave-gp6n krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280216, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:56.266636 28864 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 12:56:56.268237 28864 webserver.cc:460] Webserver started at http://127.21.235.65:43975/ using document root <none> and password file <none>
I20260809 12:56:56.269080 28864 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:56.269225 28864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:56.269517 28864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:56.272531 28864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "fd4bebf23baf4b27b55559fe1af15c49"
format_stamp: "Formatted at 2026-08-09 12:56:56 on dist-test-slave-gp6n"
server_key: "9da56a68c977490949dc6ad94037d6db"
server_key_iv: "fc25cfdf8067e68d35351774ce663a5c"
server_key_version: "encryptionkey@0"
I20260809 12:56:56.273280 28864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "fd4bebf23baf4b27b55559fe1af15c49"
format_stamp: "Formatted at 2026-08-09 12:56:56 on dist-test-slave-gp6n"
server_key: "9da56a68c977490949dc6ad94037d6db"
server_key_iv: "fc25cfdf8067e68d35351774ce663a5c"
server_key_version: "encryptionkey@0"
I20260809 12:56:56.278541 28864 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260809 12:56:56.282298 28881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:56.283697 28864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 12:56:56.283829 28864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "fd4bebf23baf4b27b55559fe1af15c49"
format_stamp: "Formatted at 2026-08-09 12:56:56 on dist-test-slave-gp6n"
server_key: "9da56a68c977490949dc6ad94037d6db"
server_key_iv: "fc25cfdf8067e68d35351774ce663a5c"
server_key_version: "encryptionkey@0"
I20260809 12:56:56.284011 28864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:56.341743 28864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:56.346343 28864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:56.346630 28864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:56.347568 28864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:56.348893 28864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:56.348968 28864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:56.349051 28864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:56.349094 28864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:56.384107 28864 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:43963
I20260809 12:56:56.384159 29006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:43963 every 8 connection(s)
I20260809 12:56:56.385506 28864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:56:56.386595 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 28864
I20260809 12:56:56.386808 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:56:56.387596 22445 external_mini_cluster.cc:1428] Setting key b78f4042e35d632363f640f36a1dfcf1
Aug 09 12:56:56 dist-test-slave-gp6n krb5kdc[28745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280216, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:56.406608 29009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:56.388508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:50623 (local address 127.21.235.126:46281)
0809 12:56:56.388963 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:56.388968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:56.390296 (+ 1328us) server_negotiation.cc:407] Connection header received
0809 12:56:56.392226 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:56.392232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:56.392324 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:56.392449 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:56.393999 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:56.394810 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:56.396438 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:56.396840 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:56.396984 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:56.400705 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:56.400710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:56.400713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:56.400745 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:56.403500 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:56.404208 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:56.404214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:56.404218 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:56.404382 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:56.404833 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:56.404841 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:56.404845 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:56.405112 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:56.405271 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:56.405958 (+ 687us) server_negotiation.cc:299] Negotiation successful
0809 12:56:56.406198 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":223,"threads_started":1}
I20260809 12:56:56.407577 29010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:56.389236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:46281 (local address 127.21.235.65:50623)
0809 12:56:56.390128 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:56.390187 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:56.391962 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:56.392708 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:56.392717 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:56.393242 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:56.393827 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:56.393845 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:56.394970 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:56.394976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:56.396273 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:56.396285 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:56.397076 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:56.397085 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:56.397325 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:56.398018 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:56.398042 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:56.400487 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:56.403631 (+ 3144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:56.403639 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:56.403655 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:56.404082 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:56.404514 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:56.404519 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:56.404523 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:56.404718 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:56.405243 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:56.405250 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:56.405583 (+ 333us) client_negotiation.cc:769] Sending connection context
0809 12:56:56.405853 (+ 270us) client_negotiation.cc:240] Negotiation successful
0809 12:56:56.406454 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":354,"threads_started":1}
I20260809 12:56:56.409161 29007 heartbeater.cc:347] Connected to a master server at 127.21.235.126:46281
I20260809 12:56:56.409516 29007 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:56.410252 29007 heartbeater.cc:511] Master 127.21.235.126:46281 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.235.66@KRBTEST.COM; defaulting to no policy
I20260809 12:56:56.412463 28796 ts_manager.cc:194] Registered new tserver with Master: fd4bebf23baf4b27b55559fe1af15c49 (127.21.235.65:43963)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.66@KRBTEST.COM" created.
I20260809 12:56:56.414110 28796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:50623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
I20260809 12:56:56.442217 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:46281
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:42091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:56.776757 29014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:56.777019 29014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:56.777112 29014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:56.784578 29014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:56.784693 29014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:56.784809 29014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:56:56.794746 29014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:42091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:46281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:56.795998 29014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:56.797286 29014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:56.810475 29020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:56.810523 29019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:56.810437 29022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:56.811574 29014 server_base.cc:1034] running on GCE node
I20260809 12:56:56.812112 29014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:56.812978 29014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:56.814179 29014 hybrid_clock.cc:648] HybridClock initialized: now 1786280216814130 us; error 40 us; skew 500 ppm
I20260809 12:56:56.814561 29014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:56 dist-test-slave-gp6n krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280216, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:56.817621 29014 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 12:56:56.819105 29014 webserver.cc:460] Webserver started at http://127.21.235.66:41181/ using document root <none> and password file <none>
I20260809 12:56:56.819736 29014 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:56.819839 29014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:56.820096 29014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:56.823112 29014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "293daa49cbec4aef89dfe70eacb34cb6"
format_stamp: "Formatted at 2026-08-09 12:56:56 on dist-test-slave-gp6n"
server_key: "3ef1dd7a50b21d448ec86d8536debd6f"
server_key_iv: "2ef342a2bb4a68c7a19a4c16d2cee30c"
server_key_version: "encryptionkey@0"
I20260809 12:56:56.823972 29014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "293daa49cbec4aef89dfe70eacb34cb6"
format_stamp: "Formatted at 2026-08-09 12:56:56 on dist-test-slave-gp6n"
server_key: "3ef1dd7a50b21d448ec86d8536debd6f"
server_key_iv: "2ef342a2bb4a68c7a19a4c16d2cee30c"
server_key_version: "encryptionkey@0"
I20260809 12:56:56.829330 29014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:56:56.832821 29031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:56.834250 29014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 12:56:56.834434 29014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "293daa49cbec4aef89dfe70eacb34cb6"
format_stamp: "Formatted at 2026-08-09 12:56:56 on dist-test-slave-gp6n"
server_key: "3ef1dd7a50b21d448ec86d8536debd6f"
server_key_iv: "2ef342a2bb4a68c7a19a4c16d2cee30c"
server_key_version: "encryptionkey@0"
I20260809 12:56:56.834649 29014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:56.849211 29014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:56.853677 29014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:56.853968 29014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:56.855036 29014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:56.856460 29014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:56.856539 29014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:56.856702 29014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:56.856770 29014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:56.891866 29014 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:35415
I20260809 12:56:56.891957 29157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:35415 every 8 connection(s)
I20260809 12:56:56.893148 29014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:56:56.895381 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 29014
I20260809 12:56:56.895601 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:56:56.896188 22445 external_mini_cluster.cc:1428] Setting key 14dbf7507a98376ea4e247af1cf49745
Aug 09 12:56:56 dist-test-slave-gp6n krb5kdc[28745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280216, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:56.914657 29009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:56.896121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:52431 (local address 127.21.235.126:46281)
0809 12:56:56.896379 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:56.896386 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:56.897653 (+ 1267us) server_negotiation.cc:407] Connection header received
0809 12:56:56.899555 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:56.899565 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:56.899668 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:56.899787 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:56.901296 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:56.902598 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:56.904193 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:56.904644 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:56.904781 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:56.908653 (+ 3872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:56.908659 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:56.908662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:56.908698 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:56.911523 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:56.912272 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:56.912279 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:56.912282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:56.912421 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:56.912881 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:56.912886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:56.912890 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:56.913169 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:56.913330 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:56.914015 (+ 685us) server_negotiation.cc:299] Negotiation successful
0809 12:56:56.914265 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 12:56:56.915649 29160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:56.896627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:46281 (local address 127.21.235.66:52431)
0809 12:56:56.897477 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:56.897530 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:56.899280 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:56.899998 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:56.900006 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:56.900522 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:56.901096 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:56.901114 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:56.902762 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:56.902766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:56.904041 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:56.904052 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:56.904889 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:56.904896 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:56.905134 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:56.905869 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:56.905892 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:56.908391 (+ 2499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:56.911683 (+ 3292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:56.911692 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:56.911709 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:56.912145 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:56.912538 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:56.912544 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:56.912548 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:56.912756 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:56.913312 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:56.913320 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:56.913657 (+ 337us) client_negotiation.cc:769] Sending connection context
0809 12:56:56.913929 (+ 272us) client_negotiation.cc:240] Negotiation successful
0809 12:56:56.914543 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":387,"threads_started":1}
I20260809 12:56:56.917339 29158 heartbeater.cc:347] Connected to a master server at 127.21.235.126:46281
I20260809 12:56:56.917662 29158 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:56.918469 29158 heartbeater.cc:511] Master 127.21.235.126:46281 requested a full tablet report, sending...
I20260809 12:56:56.920102 28796 ts_manager.cc:194] Registered new tserver with Master: 293daa49cbec4aef89dfe70eacb34cb6 (127.21.235.66:35415)
WARNING: no policy specified for kudu/127.21.235.67@KRBTEST.COM; defaulting to no policy
I20260809 12:56:56.921038 28796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:52431
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
I20260809 12:56:56.954265 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:46281
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:42091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:57.292176 29164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:57.292507 29164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:57.292689 29164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:57.300279 29164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:57.300449 29164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:57.300678 29164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:56:57.310822 29164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:42091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:46281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:57.312152 29164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:57.313592 29164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:57.328779 29170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:57.328678 29172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:57.328711 29169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:57.330044 29164 server_base.cc:1034] running on GCE node
I20260809 12:56:57.330840 29164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:57.331799 29164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:57.333065 29164 hybrid_clock.cc:648] HybridClock initialized: now 1786280217333003 us; error 63 us; skew 500 ppm
I20260809 12:56:57.333464 29164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:57 dist-test-slave-gp6n krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280217, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:57.336906 29164 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 12:56:57.338554 29164 webserver.cc:460] Webserver started at http://127.21.235.67:38453/ using document root <none> and password file <none>
I20260809 12:56:57.339533 29164 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:57.339679 29164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:57.340018 29164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:57.343216 29164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "c343bcc4a8b54b3bae1a350b3c12706a"
format_stamp: "Formatted at 2026-08-09 12:56:57 on dist-test-slave-gp6n"
server_key: "bbfa208a69d40a3f56f2d0e4fa1f157b"
server_key_iv: "1ddd6214d7154d3ec14f7d9deef25210"
server_key_version: "encryptionkey@0"
I20260809 12:56:57.344102 29164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "c343bcc4a8b54b3bae1a350b3c12706a"
format_stamp: "Formatted at 2026-08-09 12:56:57 on dist-test-slave-gp6n"
server_key: "bbfa208a69d40a3f56f2d0e4fa1f157b"
server_key_iv: "1ddd6214d7154d3ec14f7d9deef25210"
server_key_version: "encryptionkey@0"
I20260809 12:56:57.349658 29164 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:56:57.354548 29179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:57.356611 29164 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260809 12:56:57.356783 29164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "c343bcc4a8b54b3bae1a350b3c12706a"
format_stamp: "Formatted at 2026-08-09 12:56:57 on dist-test-slave-gp6n"
server_key: "bbfa208a69d40a3f56f2d0e4fa1f157b"
server_key_iv: "1ddd6214d7154d3ec14f7d9deef25210"
server_key_version: "encryptionkey@0"
I20260809 12:56:57.357043 29164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:57.388075 29164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:57.392763 29164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:57.393085 29164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:57.394076 29164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:56:57.395512 29164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:56:57.395588 29164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:57.395763 29164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:56:57.395852 29164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:57.417220 29007 heartbeater.cc:503] Master 127.21.235.126:46281 was elected leader, sending a full tablet report...
I20260809 12:56:57.431217 29164 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:33947
I20260809 12:56:57.431267 29303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:33947 every 8 connection(s)
I20260809 12:56:57.432556 29164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:56:57.438354 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 29164
I20260809 12:56:57.438568 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:56:57.439019 22445 external_mini_cluster.cc:1428] Setting key 91d00aa043fe20157cd8faced0353f51
Aug 09 12:56:57 dist-test-slave-gp6n krb5kdc[28745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280217, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:56:57.453054 29307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:57.435082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:47721 (local address 127.21.235.126:46281)
0809 12:56:57.435617 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:57.435622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:57.436434 (+ 812us) server_negotiation.cc:407] Connection header received
0809 12:56:57.438347 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:57.438354 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:57.438478 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:57.438615 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:57.440393 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:57.441209 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:57.442858 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:57.443254 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:57.443396 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:57.447117 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:57.447123 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:57.447126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:57.447159 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:57.449918 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:57.450738 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:57.450742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:57.450744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:57.450860 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:57.451316 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:57.451320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:57.451321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:57.451612 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:57.451724 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:57.452415 (+ 691us) server_negotiation.cc:299] Negotiation successful
0809 12:56:57.452630 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":306,"threads_started":1}
I20260809 12:56:57.454146 29308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:57.435523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:46281 (local address 127.21.235.67:47721)
0809 12:56:57.436218 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0809 12:56:57.436265 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 12:56:57.438037 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:56:57.438864 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:56:57.438872 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:56:57.439506 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:56:57.440204 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:57.440236 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:57.441376 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:57.441382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:57.442709 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:56:57.442722 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:56:57.443500 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:56:57.443508 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:56:57.443734 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:57.444365 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:56:57.444389 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:56:57.446917 (+ 2528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:56:57.450115 (+ 3198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:57.450122 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:57.450134 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:57.450635 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:57.450954 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:56:57.450957 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:56:57.450959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:56:57.451203 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:56:57.451727 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:56:57.451732 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:56:57.452050 (+ 318us) client_negotiation.cc:769] Sending connection context
0809 12:56:57.452268 (+ 218us) client_negotiation.cc:240] Negotiation successful
0809 12:56:57.452793 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":297,"threads_started":1}
I20260809 12:56:57.455653 29304 heartbeater.cc:347] Connected to a master server at 127.21.235.126:46281
I20260809 12:56:57.455937 29304 heartbeater.cc:464] Registering TS with master...
I20260809 12:56:57.456645 29304 heartbeater.cc:511] Master 127.21.235.126:46281 requested a full tablet report, sending...
I20260809 12:56:57.458094 28796 ts_manager.cc:194] Registered new tserver with Master: c343bcc4a8b54b3bae1a350b3c12706a (127.21.235.67:33947)
I20260809 12:56:57.458962 28796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:47721
I20260809 12:56:57.466970 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/test-admin.keytab.
Aug 09 12:56:57 dist-test-slave-gp6n krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:57.497956 22445 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 12:56:57.503706 29315 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:56:57.500856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60218 (local address 127.21.235.65:43963)
0809 12:56:57.501544 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:57.501553 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:57.501587 (+ 34us) server_negotiation.cc:407] Connection header received
0809 12:56:57.501682 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:57.501689 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:57.501903 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:57.502100 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:57.503217 (+ 1117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60218: BlockingRecv error: recv got EOF from 127.0.0.1:60218 (error 108)
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":286,"threads_started":1}
W20260809 12:56:57.503846 22445 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/krb5cc' of type FILE
Aug 09 12:56:57 dist-test-slave-gp6n krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:57.505683 22445 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 12:56:57 dist-test-slave-gp6n krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:57.507236 22445 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 12:56:57 dist-test-slave-gp6n krb5kdc[28745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 12:56:57.522646 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:57.508986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60230 (local address 127.21.235.65:43963)
0809 12:56:57.509190 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:57.509194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:57.509318 (+ 124us) server_negotiation.cc:407] Connection header received
0809 12:56:57.509459 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:57.509463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:57.509524 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:57.509607 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:57.510690 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:57.511674 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:57.512879 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:57.513448 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:57.513562 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:57.517138 (+ 3576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:57.517149 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:57.517164 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:57.517203 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:57.519911 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:57.520574 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:57.520581 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:57.520583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:57.520706 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:57.521033 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:57.521036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:57.521038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:57.521368 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:57.521577 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:57.522046 (+ 469us) server_negotiation.cc:299] Negotiation successful
0809 12:56:57.522242 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 12:56:57.528797 29315 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:56:57.527016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60234 (local address 127.21.235.65:43963)
0809 12:56:57.527211 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:57.527216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:57.527395 (+ 179us) server_negotiation.cc:407] Connection header received
0809 12:56:57.527551 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:57.527555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:57.527607 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:57.527683 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:57.528585 (+ 902us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60234: BlockingRecv error: recv got EOF from 127.0.0.1:60234 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 12:56:57.529071 22445 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/krb5cc' of type FILE
Aug 09 12:56:57 dist-test-slave-gp6n krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:57.530714 22445 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 12:56:57 dist-test-slave-gp6n krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:57.532085 22445 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 12:56:57 dist-test-slave-gp6n krb5kdc[28745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 12:56:57.546854 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:57.533432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60238 (local address 127.21.235.65:43963)
0809 12:56:57.533619 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:57.533627 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:57.533774 (+ 147us) server_negotiation.cc:407] Connection header received
0809 12:56:57.533937 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:57.533941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:57.534002 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:57.534073 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:57.535079 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:57.535743 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:57.536904 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:57.537341 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:57.537437 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:57.541461 (+ 4024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:57.541465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:57.541469 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:57.541529 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:57.544393 (+ 2864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:57.545092 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:57.545095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:57.545098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:57.545219 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:57.545594 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:57.545597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:57.545600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:57.545861 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:57.546053 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:57.546299 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 12:56:57.546540 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 12:56:57.551396 29315 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:56:57.549571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60252 (local address 127.21.235.65:43963)
0809 12:56:57.549735 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:57.549738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:57.550053 (+ 315us) server_negotiation.cc:407] Connection header received
0809 12:56:57.550174 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:57.550177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:57.550240 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:57.550314 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:57.551131 (+ 817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60252: BlockingRecv error: recv got EOF from 127.0.0.1:60252 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 12:56:57.551637 22445 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280163688078-22445-0/krb5kdc/krb5cc' of type FILE
Aug 09 12:56:57 dist-test-slave-gp6n krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:57.553242 22445 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 12:56:57 dist-test-slave-gp6n krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:57.554656 22445 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 12:56:57 dist-test-slave-gp6n krb5kdc[28745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 12:56:57.568663 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:57.556023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60256 (local address 127.21.235.65:43963)
0809 12:56:57.556228 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:57.556234 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:57.556457 (+ 223us) server_negotiation.cc:407] Connection header received
0809 12:56:57.556662 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:57.556666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:57.556719 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:57.556817 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:57.557776 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:57.558573 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:57.559758 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:57.560275 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:57.560406 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:57.563989 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:57.563993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:57.563996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:57.564027 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:57.566481 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:57.567037 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:57.567040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:57.567042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:57.567191 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:57.567495 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:57.567497 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:57.567499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:57.567743 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:57.567967 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:57.568140 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 12:56:57.568321 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 12:56:57.570291 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 28864
I20260809 12:56:57.581745 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:57.571030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60264 (local address 127.21.235.65:43963)
0809 12:56:57.571240 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:57.571246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:57.571477 (+ 231us) server_negotiation.cc:407] Connection header received
0809 12:56:57.571615 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:57.571619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:57.571672 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:57.571744 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:57.572585 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:57.573270 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:57.574480 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:57.574904 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:57.574996 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:57.576924 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:57.576928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:57.576931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:57.576959 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:57.579425 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:57.580047 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:57.580051 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:57.580053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:57.580217 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:57.580551 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:57.580554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:57.580556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:57.580793 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:57.580997 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:57.581239 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 12:56:57.581416 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 12:56:57.582353 29002 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:57.668241 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 28864
I20260809 12:56:57.688561 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 29014
Aug 09 12:56:57 dist-test-slave-gp6n krb5kdc[28745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.66@KRBTEST.COM
I20260809 12:56:57.704419 29330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:57.689469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35478 (local address 127.21.235.66:35415)
0809 12:56:57.689985 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:57.689992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:57.690016 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:56:57.690077 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:57.690084 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:57.690299 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:57.690519 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:57.691532 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:57.692552 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:57.693792 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:57.694336 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:57.694563 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:57.698350 (+ 3787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:57.698361 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:57.698382 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:57.698466 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:57.701233 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:57.701956 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:57.701966 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:57.701970 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:57.702108 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:57.702709 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:57.702716 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:57.702719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:57.703098 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:57.703445 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:57.703807 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 12:56:57.704077 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":266,"threads_started":1}
I20260809 12:56:57.705838 29153 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:57.788837 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 29014
I20260809 12:56:57.807101 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 29164
Aug 09 12:56:57 dist-test-slave-gp6n krb5kdc[28745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.67@KRBTEST.COM
I20260809 12:56:57.823124 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:57.808179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60580 (local address 127.21.235.67:33947)
0809 12:56:57.808822 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:57.808830 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:57.808857 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:56:57.808916 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:57.808923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:57.809144 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:57.809344 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:57.810519 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:57.811566 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:57.812855 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:57.813430 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:57.813581 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:57.817621 (+ 4040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:57.817632 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:57.817647 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:57.817680 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:57.820383 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:57.821074 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:57.821081 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:57.821083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:57.821204 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:57.821585 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:57.821591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:57.821594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:57.821932 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:57.822253 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:57.822568 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 12:56:57.822825 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":338,"threads_started":1}
I20260809 12:56:57.824430 29299 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:57.908272 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 29164
I20260809 12:56:57.927726 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 28759
I20260809 12:56:57.929037 28831 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:56:58.033260 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 28759
2026-08-09T12:56:58Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4274 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:56:58 dist-test-slave-gp6n krb5kdc[29338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:56:58 dist-test-slave-gp6n krb5kdc[29338](info): set up 2 sockets
Aug 09 12:56:58 dist-test-slave-gp6n krb5kdc[29338](info): commencing operation
krb5kdc: starting...
W20260809 12:56:59.205664 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.004s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:56:59 dist-test-slave-gp6n krb5kdc[29338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:56:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:56:59Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.126.keytab.
Entry for principal oryx/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.126.keytab.
I20260809 12:56:59.446290 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:33521
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.126.keytab
--principal=oryx/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:44997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:33521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:56:59.773475 29357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:56:59.773819 29357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:56:59.773984 29357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:56:59.781360 29357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:56:59.781507 29357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:56:59.781622 29357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:56:59.781730 29357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:56:59.781808 29357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:56:59.791730 29357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:44997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:33521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.126.keytab
--principal=oryx/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:33521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:56:59.793087 29357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:56:59.794461 29357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:56:59.806455 29366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:59.806455 29374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:56:59.806455 29365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:56:59.807636 29357 server_base.cc:1034] running on GCE node
I20260809 12:56:59.808424 29357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:56:59.809862 29357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:56:59.811100 29357 hybrid_clock.cc:648] HybridClock initialized: now 1786280219811046 us; error 48 us; skew 500 ppm
I20260809 12:56:59.811446 29357 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:56:59 dist-test-slave-gp6n krb5kdc[29338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280219, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:56:59.814764 29357 init.cc:377] Logged in from keytab as oryx/127.21.235.126@KRBTEST.COM (short username oryx)
I20260809 12:56:59.816198 29357 webserver.cc:460] Webserver started at http://127.21.235.126:32845/ using document root <none> and password file <none>
I20260809 12:56:59.816843 29357 fs_manager.cc:359] Metadata directory not provided
I20260809 12:56:59.816993 29357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:56:59.817307 29357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:56:59.820313 29357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "e0475d10a6054e859f62c2497daa235c"
format_stamp: "Formatted at 2026-08-09 12:56:59 on dist-test-slave-gp6n"
server_key: "74b58ea9f488e37f06cefe462fa3b30d"
server_key_iv: "318f75ffbf094dddd5e7618642ffa7ff"
server_key_version: "encryptionkey@0"
I20260809 12:56:59.821048 29357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "e0475d10a6054e859f62c2497daa235c"
format_stamp: "Formatted at 2026-08-09 12:56:59 on dist-test-slave-gp6n"
server_key: "74b58ea9f488e37f06cefe462fa3b30d"
server_key_iv: "318f75ffbf094dddd5e7618642ffa7ff"
server_key_version: "encryptionkey@0"
I20260809 12:56:59.826220 29357 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:56:59.829583 29382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:56:59.830961 29357 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:56:59.831151 29357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "e0475d10a6054e859f62c2497daa235c"
format_stamp: "Formatted at 2026-08-09 12:56:59 on dist-test-slave-gp6n"
server_key: "74b58ea9f488e37f06cefe462fa3b30d"
server_key_iv: "318f75ffbf094dddd5e7618642ffa7ff"
server_key_version: "encryptionkey@0"
I20260809 12:56:59.831344 29357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:56:59.846096 29357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:56:59.850806 29357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:56:59.851119 29357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:56:59.870100 29357 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:33521
I20260809 12:56:59.870131 29442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:33521 every 8 connection(s)
I20260809 12:56:59.871501 29357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:56:59.875349 29443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:56:59.878436 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 29357
I20260809 12:56:59.878648 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:56:59.879537 22445 external_mini_cluster.cc:1428] Setting key 5e9fa483dea2c9552ce4d46c05899927
I20260809 12:56:59.888439 29443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c: Bootstrap starting.
I20260809 12:56:59.892166 29443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c: Neither blocks nor log segments found. Creating new log.
I20260809 12:56:59.893481 29443 log.cc:824] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c: Log is configured to *not* fsync() on all Append() calls
Aug 09 12:56:59 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM
I20260809 12:56:59.896384 29443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c: No bootstrap required, opened a new log
I20260809 12:56:59.904486 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:56:59.881757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50186 (local address 127.21.235.126:33521)
0809 12:56:59.882557 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0809 12:56:59.882569 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:56:59.882611 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:56:59.884242 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:56:59.884264 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:56:59.884642 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:56:59.884980 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:56:59.886784 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:59.888089 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:59.889832 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:56:59.890606 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:56:59.890736 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:56:59.895698 (+ 4962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:56:59.895706 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:56:59.895720 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:56:59.895757 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:56:59.899816 (+ 4059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:59.900500 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:59.900510 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:59.900522 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:59.900734 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:56:59.901179 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:56:59.901184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:56:59.901188 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:56:59.901879 (+ 691us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:56:59.902227 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0809 12:56:59.902668 (+ 441us) server_negotiation.cc:299] Negotiation successful
0809 12:56:59.903132 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":366,"threads_started":1}
I20260809 12:56:59.906692 29443 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } }
I20260809 12:56:59.907038 29443 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:56:59.907171 29443 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0475d10a6054e859f62c2497daa235c, State: Initialized, Role: FOLLOWER
I20260809 12:56:59.907716 29443 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } }
I20260809 12:56:59.907951 29443 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:56:59.908093 29443 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:56:59.908267 29443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:56:59.910353 29443 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } }
I20260809 12:56:59.910882 29443 leader_election.cc:302] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0475d10a6054e859f62c2497daa235c; no voters:
I20260809 12:56:59.911458 29443 leader_election.cc:288] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:56:59.911573 29448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:56:59.912351 29448 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [term 1 LEADER]: Becoming Leader. State: Replica: e0475d10a6054e859f62c2497daa235c, State: Running, Role: LEADER
I20260809 12:56:59.912885 29448 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } }
I20260809 12:56:59.913568 29443 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:56:59.916054 29450 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0475d10a6054e859f62c2497daa235c. Latest consensus state: current_term: 1 leader_uuid: "e0475d10a6054e859f62c2497daa235c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } }
I20260809 12:56:59.916296 29449 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0475d10a6054e859f62c2497daa235c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } }
I20260809 12:56:59.916632 29449 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:59.916498 29450 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: This master's current role is: LEADER
I20260809 12:56:59.917901 29456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:56:59.924548 29456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:56:59.935197 29456 catalog_manager.cc:1347] Generated new cluster ID: 9ea99b7009ef49d794f2c37b68465cb1
I20260809 12:56:59.935319 29456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:56:59.945819 29456 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:56:59.947786 29456 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:56:59.955628 29456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c: Generated new TSK 0
I20260809 12:56:59.956619 29456 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.65.keytab.
Entry for principal oryx/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.65.keytab.
I20260809 12:57:00.019028 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:33521
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.65.keytab
--principal=oryx/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:44997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:57:00.359612 29474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:00.359884 29474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:00.359978 29474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:00.367097 29474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:00.367210 29474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:00.367324 29474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:57:00.377002 29474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:44997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.65.keytab
--principal=oryx/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:33521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:00.378247 29474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:00.379644 29474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:00.393275 29486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:00.393357 29483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:00.393332 29481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:00.394193 29474 server_base.cc:1034] running on GCE node
I20260809 12:57:00.394840 29474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:00.395783 29474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:00.397027 29474 hybrid_clock.cc:648] HybridClock initialized: now 1786280220396974 us; error 42 us; skew 500 ppm
I20260809 12:57:00.397341 29474 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:00 dist-test-slave-gp6n krb5kdc[29338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280220, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:00.400570 29474 init.cc:377] Logged in from keytab as oryx/127.21.235.65@KRBTEST.COM (short username oryx)
I20260809 12:57:00.401979 29474 webserver.cc:460] Webserver started at http://127.21.235.65:38813/ using document root <none> and password file <none>
I20260809 12:57:00.402724 29474 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:00.402882 29474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:00.403194 29474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:00.406109 29474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "e2e31488363744cf90ae9aac7c1883d9"
format_stamp: "Formatted at 2026-08-09 12:57:00 on dist-test-slave-gp6n"
server_key: "e3fde278a7289fd250d4f4c200afdc78"
server_key_iv: "24ea9a2399d91a076c9fc3c55322cab9"
server_key_version: "encryptionkey@0"
I20260809 12:57:00.406962 29474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "e2e31488363744cf90ae9aac7c1883d9"
format_stamp: "Formatted at 2026-08-09 12:57:00 on dist-test-slave-gp6n"
server_key: "e3fde278a7289fd250d4f4c200afdc78"
server_key_iv: "24ea9a2399d91a076c9fc3c55322cab9"
server_key_version: "encryptionkey@0"
I20260809 12:57:00.412354 29474 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:57:00.416011 29493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:00.417618 29474 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:57:00.417801 29474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "e2e31488363744cf90ae9aac7c1883d9"
format_stamp: "Formatted at 2026-08-09 12:57:00 on dist-test-slave-gp6n"
server_key: "e3fde278a7289fd250d4f4c200afdc78"
server_key_iv: "24ea9a2399d91a076c9fc3c55322cab9"
server_key_version: "encryptionkey@0"
I20260809 12:57:00.418006 29474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:00.431336 29474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:00.435670 29474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:00.436036 29474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:00.437036 29474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:00.438433 29474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:00.438547 29474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:00.438701 29474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:00.438781 29474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:00.474584 29474 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:39115
I20260809 12:57:00.474622 29618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:39115 every 8 connection(s)
I20260809 12:57:00.475898 29474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:57:00.481602 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 29474
I20260809 12:57:00.481818 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:57:00.482684 22445 external_mini_cluster.cc:1428] Setting key c9d7c8528d02b5f87afedee82a85f652
Aug 09 12:57:00 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280220, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM
I20260809 12:57:00.497934 29622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:00.478573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:43295 (local address 127.21.235.126:33521)
0809 12:57:00.479010 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:00.479015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:00.479924 (+ 909us) server_negotiation.cc:407] Connection header received
0809 12:57:00.482036 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:00.482041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:00.482128 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:00.482228 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:00.484263 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:00.485244 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:00.487123 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:00.487561 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:00.487709 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:00.491766 (+ 4057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:00.491772 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:00.491775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:00.491811 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:00.494676 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:00.495451 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:00.495459 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:00.495468 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:00.495588 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:00.496057 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:00.496067 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:00.496070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:00.496340 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:00.496509 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:00.497277 (+ 768us) server_negotiation.cc:299] Negotiation successful
0809 12:57:00.497519 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":235,"threads_started":1}
I20260809 12:57:00.499053 29624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:00.479039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.65:43295)
0809 12:57:00.479750 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:00.479806 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:00.481696 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:00.482554 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:00.482567 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:00.483348 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:00.484031 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:00.484056 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:00.485459 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:00.485464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:00.486951 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:00.486964 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:00.487812 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:00.487820 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:00.488085 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:00.488826 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:00.488850 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:00.491511 (+ 2661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:00.494826 (+ 3315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:00.494835 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:00.494854 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:00.495311 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:00.495710 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:00.495715 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:00.495719 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:00.495923 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:00.496486 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:00.496494 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:00.496860 (+ 366us) client_negotiation.cc:769] Sending connection context
0809 12:57:00.497140 (+ 280us) client_negotiation.cc:240] Negotiation successful
0809 12:57:00.497737 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":297,"threads_started":1}
I20260809 12:57:00.500756 29619 heartbeater.cc:347] Connected to a master server at 127.21.235.126:33521
I20260809 12:57:00.501132 29619 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:00.502106 29619 heartbeater.cc:511] Master 127.21.235.126:33521 requested a full tablet report, sending...
I20260809 12:57:00.504966 29403 ts_manager.cc:194] Registered new tserver with Master: e2e31488363744cf90ae9aac7c1883d9 (127.21.235.65:39115)
WARNING: no policy specified for oryx/127.21.235.66@KRBTEST.COM; defaulting to no policy
I20260809 12:57:00.507380 29403 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:43295
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.66.keytab.
Entry for principal oryx/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.66.keytab.
I20260809 12:57:00.538178 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:33521
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.66.keytab
--principal=oryx/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:44997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:57:00.871347 29628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:00.871635 29628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:00.871793 29628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:00.879078 29628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:00.879179 29628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:00.879422 29628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:57:00.888875 29628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:44997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.66.keytab
--principal=oryx/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:33521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:00.890208 29628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:00.891615 29628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:00.904965 29635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:00.905038 29633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:00.905063 29637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:00.905828 29628 server_base.cc:1034] running on GCE node
I20260809 12:57:00.906575 29628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:00.907584 29628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:00.908797 29628 hybrid_clock.cc:648] HybridClock initialized: now 1786280220908750 us; error 34 us; skew 500 ppm
I20260809 12:57:00.909137 29628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:00 dist-test-slave-gp6n krb5kdc[29338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280220, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:00.916035 29628 init.cc:377] Logged in from keytab as oryx/127.21.235.66@KRBTEST.COM (short username oryx)
I20260809 12:57:00.917658 29628 webserver.cc:460] Webserver started at http://127.21.235.66:46397/ using document root <none> and password file <none>
I20260809 12:57:00.918433 29628 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:00.918642 29628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:00.918991 29628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:00.922267 29628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "86e19c19da6246d89330b6a6f23b0c50"
format_stamp: "Formatted at 2026-08-09 12:57:00 on dist-test-slave-gp6n"
server_key: "af4d3583cb135d0be464dc9ce1cb5a18"
server_key_iv: "0f85efbd76d5dd36b6afa9885fbdeadf"
server_key_version: "encryptionkey@0"
I20260809 12:57:00.923237 29628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "86e19c19da6246d89330b6a6f23b0c50"
format_stamp: "Formatted at 2026-08-09 12:57:00 on dist-test-slave-gp6n"
server_key: "af4d3583cb135d0be464dc9ce1cb5a18"
server_key_iv: "0f85efbd76d5dd36b6afa9885fbdeadf"
server_key_version: "encryptionkey@0"
I20260809 12:57:00.928821 29628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:57:00.932528 29644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:00.933885 29628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:57:00.934067 29628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "86e19c19da6246d89330b6a6f23b0c50"
format_stamp: "Formatted at 2026-08-09 12:57:00 on dist-test-slave-gp6n"
server_key: "af4d3583cb135d0be464dc9ce1cb5a18"
server_key_iv: "0f85efbd76d5dd36b6afa9885fbdeadf"
server_key_version: "encryptionkey@0"
I20260809 12:57:00.934273 29628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:00.962601 29628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:00.967144 29628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:00.967453 29628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:00.968420 29628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:00.969782 29628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:00.969890 29628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:00.970047 29628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:00.970121 29628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:01.005271 29628 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:40791
I20260809 12:57:01.005375 29767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:40791 every 8 connection(s)
I20260809 12:57:01.006686 29628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:57:01.011894 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 29628
I20260809 12:57:01.012169 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:57:01.012687 22445 external_mini_cluster.cc:1428] Setting key 85671fa9e1397721ce4ef6b6cbe17032
Aug 09 12:57:01 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280220, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM
I20260809 12:57:01.029446 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:01.009423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:45747 (local address 127.21.235.126:33521)
0809 12:57:01.009888 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:01.009894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:01.010917 (+ 1023us) server_negotiation.cc:407] Connection header received
0809 12:57:01.012924 (+ 2007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:01.012930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:01.013023 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:01.013155 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:01.015061 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:01.015790 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:01.017320 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:01.017747 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:01.017920 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:01.022280 (+ 4360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:01.022285 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:01.022288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:01.022320 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:01.025837 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:01.026724 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:01.026733 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:01.026739 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:01.026903 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:01.027364 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:01.027371 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:01.027377 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:01.027771 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:01.027926 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:01.028678 (+ 752us) server_negotiation.cc:299] Negotiation successful
0809 12:57:01.028934 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":240,"threads_started":1}
I20260809 12:57:01.030213 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:01.009962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.66:45747)
0809 12:57:01.010742 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:01.010797 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:01.012620 (+ 1823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:01.013393 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:01.013402 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:01.014122 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:01.014898 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:01.014918 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:01.015935 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:01.015942 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:01.017164 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:01.017183 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:01.017974 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:01.017983 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:01.018274 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:01.019032 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:01.019054 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:01.022036 (+ 2982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:01.025981 (+ 3945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:01.025988 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:01.026001 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:01.026588 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:01.027017 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:01.027021 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:01.027023 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:01.027240 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:01.027907 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:01.027914 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:01.028234 (+ 320us) client_negotiation.cc:769] Sending connection context
0809 12:57:01.028530 (+ 296us) client_negotiation.cc:240] Negotiation successful
0809 12:57:01.028943 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":301,"threads_started":1}
I20260809 12:57:01.031929 29768 heartbeater.cc:347] Connected to a master server at 127.21.235.126:33521
I20260809 12:57:01.032279 29768 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:01.033227 29768 heartbeater.cc:511] Master 127.21.235.126:33521 requested a full tablet report, sending...
I20260809 12:57:01.035005 29403 ts_manager.cc:194] Registered new tserver with Master: 86e19c19da6246d89330b6a6f23b0c50 (127.21.235.66:40791)
I20260809 12:57:01.036085 29403 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:45747
WARNING: no policy specified for oryx/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.67.keytab.
Entry for principal oryx/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.67.keytab.
I20260809 12:57:01.074582 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:33521
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.67.keytab
--principal=oryx/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:44997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:57:01.431012 29776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:01.431320 29776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:01.431499 29776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:01.438853 29776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:01.438995 29776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:01.439162 29776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:57:01.448602 29776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:44997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.67.keytab
--principal=oryx/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:33521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:01.449852 29776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:01.451208 29776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:01.463826 29781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:01.463898 29782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:01.463922 29784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:01.464897 29776 server_base.cc:1034] running on GCE node
I20260809 12:57:01.465479 29776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:01.466346 29776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:01.467585 29776 hybrid_clock.cc:648] HybridClock initialized: now 1786280221467547 us; error 34 us; skew 500 ppm
I20260809 12:57:01.467849 29776 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:01 dist-test-slave-gp6n krb5kdc[29338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280221, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:01.471150 29776 init.cc:377] Logged in from keytab as oryx/127.21.235.67@KRBTEST.COM (short username oryx)
I20260809 12:57:01.472587 29776 webserver.cc:460] Webserver started at http://127.21.235.67:43807/ using document root <none> and password file <none>
I20260809 12:57:01.473290 29776 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:01.473429 29776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:01.473834 29776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:01.476888 29776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "3865e20ae461466f93bdc56fa5d870b5"
format_stamp: "Formatted at 2026-08-09 12:57:01 on dist-test-slave-gp6n"
server_key: "14866281d012a68a0318ebdd072f01c4"
server_key_iv: "a9ee10fa50fcb9c4edb2c92206c8e361"
server_key_version: "encryptionkey@0"
I20260809 12:57:01.477777 29776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "3865e20ae461466f93bdc56fa5d870b5"
format_stamp: "Formatted at 2026-08-09 12:57:01 on dist-test-slave-gp6n"
server_key: "14866281d012a68a0318ebdd072f01c4"
server_key_iv: "a9ee10fa50fcb9c4edb2c92206c8e361"
server_key_version: "encryptionkey@0"
I20260809 12:57:01.483074 29776 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260809 12:57:01.486665 29791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:01.487989 29776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:57:01.488168 29776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "3865e20ae461466f93bdc56fa5d870b5"
format_stamp: "Formatted at 2026-08-09 12:57:01 on dist-test-slave-gp6n"
server_key: "14866281d012a68a0318ebdd072f01c4"
server_key_iv: "a9ee10fa50fcb9c4edb2c92206c8e361"
server_key_version: "encryptionkey@0"
I20260809 12:57:01.488370 29776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:01.500617 29776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:01.504825 29776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:01.505133 29776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:01.506101 29776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:01.507459 29776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:01.507562 29776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:01.507706 29776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:01.507777 29776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:01.510728 29619 heartbeater.cc:503] Master 127.21.235.126:33521 was elected leader, sending a full tablet report...
I20260809 12:57:01.543274 29776 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:39729
I20260809 12:57:01.543325 29920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:39729 every 8 connection(s)
I20260809 12:57:01.544562 29776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:57:01.549794 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 29776
I20260809 12:57:01.549985 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:57:01.550593 22445 external_mini_cluster.cc:1428] Setting key 3eac48abfa388ca02932c1f72d052bee
Aug 09 12:57:01 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280221, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM
I20260809 12:57:01.565490 29924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:01.547170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:57007 (local address 127.21.235.126:33521)
0809 12:57:01.547629 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:01.547635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:01.548740 (+ 1105us) server_negotiation.cc:407] Connection header received
0809 12:57:01.551248 (+ 2508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:01.551255 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:01.551359 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:01.551498 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:01.553471 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:01.554381 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:01.555971 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:01.556384 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:01.556483 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:01.559732 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:01.559737 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:01.559741 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:01.559774 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:01.562311 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:01.562991 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:01.562995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:01.562997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:01.563117 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:01.563546 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:01.563550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:01.563553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:01.563863 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:01.564009 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:01.564844 (+ 835us) server_negotiation.cc:299] Negotiation successful
0809 12:57:01.565120 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":215,"threads_started":1}
I20260809 12:57:01.566602 29925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:01.547777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.67:57007)
0809 12:57:01.548478 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:01.548524 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:01.550857 (+ 2333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:01.551763 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:01.551772 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:01.552593 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:01.553292 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:01.553310 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:01.554592 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:01.554596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:01.555837 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:01.555848 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:01.556567 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:01.556571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:01.556770 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:01.557408 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:01.557429 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:01.559546 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:01.562472 (+ 2926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:01.562478 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:01.562490 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:01.562883 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:01.563208 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:01.563211 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:01.563213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:01.563382 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:01.563982 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:01.563987 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:01.564371 (+ 384us) client_negotiation.cc:769] Sending connection context
0809 12:57:01.564697 (+ 326us) client_negotiation.cc:240] Negotiation successful
0809 12:57:01.565228 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":266,"threads_started":1}
I20260809 12:57:01.567811 29921 heartbeater.cc:347] Connected to a master server at 127.21.235.126:33521
I20260809 12:57:01.568154 29921 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:01.568936 29921 heartbeater.cc:511] Master 127.21.235.126:33521 requested a full tablet report, sending...
I20260809 12:57:01.570279 29403 ts_manager.cc:194] Registered new tserver with Master: 3865e20ae461466f93bdc56fa5d870b5 (127.21.235.67:39729)
I20260809 12:57:01.571117 29403 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:57007
I20260809 12:57:01.578774 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.235.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.235.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.235.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.125.keytab.
Entry for principal oryx/127.21.235.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.125.keytab.
I20260809 12:57:01.638250 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.125:42291
--webserver_interface=127.21.235.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.125.keytab
--principal=oryx/127.21.235.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:44997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:33521,127.21.235.125:42291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:57:01.976400 29929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:01.976688 29929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:01.976876 29929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:01.984337 29929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:57:01.984447 29929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:01.984561 29929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:01.984666 29929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:57:01.984745 29929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:57:01.994943 29929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:44997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:33521,127.21.235.125:42291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.125.keytab
--principal=oryx/127.21.235.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.125:42291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.235.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
W20260809 12:57:01.995313 29929 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.21.235.126:33521,127.21.235.125:42291'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 12:57:01.996400 29929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:01.997732 29929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:02.009644 29942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:02.009677 29940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:02.010072 29939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:02.011052 29929 server_base.cc:1034] running on GCE node
I20260809 12:57:02.011749 29929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:02.013018 29929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:02.014259 29929 hybrid_clock.cc:648] HybridClock initialized: now 1786280222014203 us; error 31 us; skew 500 ppm
I20260809 12:57:02.014724 29929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:02 dist-test-slave-gp6n krb5kdc[29338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280222, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:02.018112 29929 init.cc:377] Logged in from keytab as oryx/127.21.235.125@KRBTEST.COM (short username oryx)
I20260809 12:57:02.019996 29929 webserver.cc:460] Webserver started at http://127.21.235.125:45353/ using document root <none> and password file <none>
I20260809 12:57:02.020735 29929 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:02.020856 29929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:02.021199 29929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:02.024420 29929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/data/instance:
uuid: "91d3abc7c1254b84bb14a33b01d586d1"
format_stamp: "Formatted at 2026-08-09 12:57:02 on dist-test-slave-gp6n"
server_key: "e3cc8c3bd032d4f02faea397948a3f32"
server_key_iv: "73d00ccc988121278d453cf3ae5744e4"
server_key_version: "encryptionkey@0"
I20260809 12:57:02.025213 29929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/wal/instance:
uuid: "91d3abc7c1254b84bb14a33b01d586d1"
format_stamp: "Formatted at 2026-08-09 12:57:02 on dist-test-slave-gp6n"
server_key: "e3cc8c3bd032d4f02faea397948a3f32"
server_key_iv: "73d00ccc988121278d453cf3ae5744e4"
server_key_version: "encryptionkey@0"
I20260809 12:57:02.030483 29929 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 12:57:02.033854 29949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:02.035382 29929 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:57:02.035622 29929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/wal
uuid: "91d3abc7c1254b84bb14a33b01d586d1"
format_stamp: "Formatted at 2026-08-09 12:57:02 on dist-test-slave-gp6n"
server_key: "e3cc8c3bd032d4f02faea397948a3f32"
server_key_iv: "73d00ccc988121278d453cf3ae5744e4"
server_key_version: "encryptionkey@0"
I20260809 12:57:02.035847 29929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:02.039582 29768 heartbeater.cc:503] Master 127.21.235.126:33521 was elected leader, sending a full tablet report...
I20260809 12:57:02.061771 29929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:02.065924 29929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:02.066223 29929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:02.085004 29929 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.125:42291
I20260809 12:57:02.085023 30004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.125:42291 every 8 connection(s)
I20260809 12:57:02.086472 29929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/data/info.pb
I20260809 12:57:02.090278 30005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:02.091858 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 29929
I20260809 12:57:02.099921 30005 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:57:02 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.235.125@KRBTEST.COM
Aug 09 12:57:02 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280222, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.125@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM
I20260809 12:57:02.117640 30008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.094289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53784 (local address 127.21.235.125:42291)
0809 12:57:02.095155 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:02.095170 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:02.095215 (+ 45us) server_negotiation.cc:407] Connection header received
0809 12:57:02.096785 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:02.096810 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:02.097221 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:02.097560 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:02.099498 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.100979 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.102960 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.104094 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.104224 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.109136 (+ 4912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:02.109149 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:02.109163 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:02.109209 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:02.112832 (+ 3623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.113755 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.113767 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.113776 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.113995 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.114611 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.114614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.114616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.115101 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:02.115411 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:02.115869 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 12:57:02.116305 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":373,"threads_started":1}
I20260809 12:57:02.119917 30010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.102840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.0.0.1:40710)
0809 12:57:02.103763 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:02.103809 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:02.103990 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:02.104448 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:02.104456 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:02.105041 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:02.105271 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.105285 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.106879 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.106884 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.108050 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.108069 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.109142 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.109149 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.109382 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.110109 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:02.110133 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:02.112964 (+ 2831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:02.116979 (+ 4015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.116988 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.116993 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.117414 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.117863 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.117870 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.117874 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.118082 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.119001 (+ 919us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:02.119011 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:02.119192 (+ 181us) client_negotiation.cc:769] Sending connection context
0809 12:57:02.119374 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 12:57:02.119605 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":543,"threads_started":1}
I20260809 12:57:02.120416 30009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.102871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40710 (local address 127.21.235.126:33521)
0809 12:57:02.103456 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:02.103463 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:02.103908 (+ 445us) server_negotiation.cc:407] Connection header received
0809 12:57:02.104124 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:02.104128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:02.104241 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:02.104361 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:02.105429 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.106695 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.108200 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.108856 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.108978 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.113167 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:02.113181 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:02.113185 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:02.113251 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:02.116842 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.117535 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.117540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.117543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.117707 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.118247 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.118254 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.118258 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.118713 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:02.118886 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:02.119545 (+ 659us) server_negotiation.cc:299] Negotiation successful
0809 12:57:02.119780 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":282,"threads_started":1}
I20260809 12:57:02.122224 30005 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:57:02 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280222, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.125@KRBTEST.COM for oryx/127.21.235.125@KRBTEST.COM
I20260809 12:57:02.134037 30010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.122928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:42291 (local address 127.0.0.1:53798)
0809 12:57:02.123126 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:02.123145 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:02.123278 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:02.123563 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:02.123567 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:02.123903 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:02.124025 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.124033 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.124964 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.124969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.125825 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.125836 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.126569 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.126574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.126765 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.127240 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:02.127256 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:02.129162 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:02.131795 (+ 2633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.131802 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.131806 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.132157 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.132498 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.132502 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.132506 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.132627 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.133127 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:02.133132 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:02.133271 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 12:57:02.133404 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 12:57:02.133691 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 12:57:02.134136 30008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.122979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53798 (local address 127.21.235.125:42291)
0809 12:57:02.123126 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:02.123132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:02.123220 (+ 88us) server_negotiation.cc:407] Connection header received
0809 12:57:02.123393 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:02.123396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:02.123444 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:02.123546 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:02.124152 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.124826 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.125965 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.126372 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.126564 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.129354 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:02.129359 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:02.129362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:02.129388 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:02.131671 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.132270 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.132277 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.132280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.132388 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.132726 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.132734 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.132737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.133006 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:02.133146 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:02.133475 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 12:57:02.133821 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":157,"server-negotiator.queue_time_us":45}
I20260809 12:57:02.139111 30005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: Bootstrap starting.
I20260809 12:57:02.142311 30005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:02.143371 30005 log.cc:824] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: Log is configured to *not* fsync() on all Append() calls
I20260809 12:57:02.145922 30005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: No bootstrap required, opened a new log
I20260809 12:57:02.152983 30005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } }
I20260809 12:57:02.153292 30005 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:02.153371 30005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91d3abc7c1254b84bb14a33b01d586d1, State: Initialized, Role: FOLLOWER
I20260809 12:57:02.153928 30005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } }
I20260809 12:57:02.154754 30012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } } }
I20260809 12:57:02.155054 30012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:57:02.155756 30005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 12:57:02.164880 30025 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 12:57:02.165028 30025 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 12:57:02.167769 29929 master_runner.cc:428] Detected that this master 91d3abc7c1254b84bb14a33b01d586d1 is joining an existing cluster
I20260809 12:57:02.167852 29929 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.235.125:42291
I20260809 12:57:02.196272 30009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.172321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40718 (local address 127.21.235.126:33521)
0809 12:57:02.172522 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:02.172530 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:02.175329 (+ 2799us) server_negotiation.cc:407] Connection header received
0809 12:57:02.175739 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:02.175745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:02.175916 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:02.176038 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:02.178496 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.179998 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.182772 (+ 2774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.183707 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.183891 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.187562 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:02.187578 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:02.187591 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:02.187636 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:02.192268 (+ 4632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.193444 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.193452 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.193457 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.193621 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.194123 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.194130 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.194133 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.194594 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:02.194760 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:02.195530 (+ 770us) server_negotiation.cc:299] Negotiation successful
0809 12:57:02.195816 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 12:57:02.197875 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.172321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.0.0.1:40718)
0809 12:57:02.175183 (+ 2862us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:02.175218 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:02.175481 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:02.176478 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:02.176485 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:02.177602 (+ 1117us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:02.178207 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.178225 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.180229 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.180237 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.182539 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.182564 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.184286 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.184294 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.184592 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.185337 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:02.185437 (+ 100us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:02.187138 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:02.192463 (+ 5325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.192469 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.192473 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.193266 (+ 793us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.193784 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.193791 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.193795 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.193982 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.194859 (+ 877us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:02.194870 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:02.195164 (+ 294us) client_negotiation.cc:769] Sending connection context
0809 12:57:02.196672 (+ 1508us) client_negotiation.cc:240] Negotiation successful
0809 12:57:02.197174 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2539,"mutex_wait_us":26,"thread_start_us":2334,"threads_started":1}
I20260809 12:57:02.207022 30008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.175448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53802 (local address 127.21.235.125:42291)
0809 12:57:02.175635 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:02.175641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:02.188370 (+ 12729us) server_negotiation.cc:407] Connection header received
0809 12:57:02.189075 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:02.189081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:02.189147 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:02.189219 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:02.191516 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.192680 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.194859 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.195600 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.195694 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.198336 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:02.198343 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:02.198348 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:02.198383 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:02.202627 (+ 4244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.204688 (+ 2061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.204694 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.204697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.204860 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.205286 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.205291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.205294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.205647 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:02.205826 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:02.206339 (+ 513us) server_negotiation.cc:299] Negotiation successful
0809 12:57:02.206617 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":60}
I20260809 12:57:02.208341 30032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.175745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:42291 (local address 127.0.0.1:53802)
0809 12:57:02.188273 (+ 12528us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:02.188297 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:02.188930 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:02.189425 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:02.189430 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:02.190504 (+ 1074us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:02.191313 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.191329 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.193045 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.193052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.194698 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.194713 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.195998 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.196004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.196175 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.196824 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:02.196847 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:02.198014 (+ 1167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:02.202787 (+ 4773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.202793 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.202799 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.204559 (+ 1760us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.205016 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.205020 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.205024 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.205178 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.205902 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:02.205907 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:02.206080 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 12:57:02.207596 (+ 1516us) client_negotiation.cc:240] Negotiation successful
0809 12:57:02.207900 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12374,"thread_start_us":12616,"threads_started":1}
WARNING: no policy specified for oryx/127.21.235.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.235.124@KRBTEST.COM" created.
Aug 09 12:57:02 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280219, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.126@KRBTEST.COM for oryx/127.21.235.125@KRBTEST.COM
I20260809 12:57:02.232430 30008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.205671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53810 (local address 127.21.235.125:42291)
0809 12:57:02.207562 (+ 1891us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:02.207568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:02.209992 (+ 2424us) server_negotiation.cc:407] Connection header received
0809 12:57:02.210461 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:02.210467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:02.210579 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:02.210656 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:02.212456 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.214151 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.216967 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.217830 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.217971 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.225852 (+ 7881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:02.225858 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:02.225861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:02.225897 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:02.228998 (+ 3101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.230109 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.230121 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.230124 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.230266 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.230758 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.230762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.230764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.231072 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:02.231235 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:02.231748 (+ 513us) server_negotiation.cc:299] Negotiation successful
0809 12:57:02.231989 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1796,"thread_start_us":12099,"threads_started":1}
I20260809 12:57:02.232517 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.205083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:42291 (local address 127.0.0.1:53810)
0809 12:57:02.209852 (+ 4769us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:02.209898 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:02.210150 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:02.210947 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:02.210956 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:02.211900 (+ 944us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:02.212206 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.212221 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.214522 (+ 2301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.214532 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.216766 (+ 2234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.216786 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.218335 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.218343 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.218638 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.219569 (+ 931us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:02.219601 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:02.225610 (+ 6009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:02.229151 (+ 3541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.229167 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.229175 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.229947 (+ 772us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.230393 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.230398 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.230434 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.230618 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.231214 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:02.231223 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:02.231430 (+ 207us) client_negotiation.cc:769] Sending connection context
0809 12:57:02.231641 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 12:57:02.231989 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4483,"thread_start_us":4365,"threads_started":1}
I20260809 12:57:02.235211 29403 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.235.125:42291: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1"
member_type: NON_VOTER
last_known_addr {
host: "127.21.235.125"
port: 42291
}
attrs {
promote: true
}
}
dest_uuid: "e0475d10a6054e859f62c2497daa235c"
cas_config_opid_index: -1
I20260809 12:57:02.236966 29403 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: NON_VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: true } }
I20260809 12:57:02.250874 30039 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 91d3abc7c1254b84bb14a33b01d586d1 (127.21.235.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: NON_VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: true } } }
I20260809 12:57:02.251935 29980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:02.254485 30039 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.235.125:42291
I20260809 12:57:02.255070 30039 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e0475d10a6054e859f62c2497daa235c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: NON_VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: true } } }
I20260809 12:57:02.255333 30039 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:02.255496 29929 master.cc:537] Master@127.21.235.125:42291 shutting down...
I20260809 12:57:02.260223 29980 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 1 FOLLOWER]: Refusing update from remote peer e0475d10a6054e859f62c2497daa235c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 12:57:02.262040 30039 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [LEADER]: Connected to new peer: Peer: permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: NON_VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 12:57:02.265401 29385 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 12:57:02.267539 29385 consensus_peers.cc:487] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c -> Peer 91d3abc7c1254b84bb14a33b01d586d1 (127.21.235.125:42291): Couldn't send request to peer 91d3abc7c1254b84bb14a33b01d586d1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 12:57:02.269030 29929 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:57:02.269439 29929 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:02.269656 29929 tablet_replica.cc:330] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.235.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.124.keytab.
Entry for principal oryx/127.21.235.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.124.keytab.
I20260809 12:57:02.272321 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.124:40403
--webserver_interface=127.21.235.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.124.keytab
--principal=oryx/127.21.235.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:44997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:33521,127.21.235.125:42291,127.21.235.124:40403
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:57:02.288069 29929 master.cc:552] Master@127.21.235.125:42291 shutdown complete.
I20260809 12:57:02.288224 29929 master_runner.cc:305] Clearing existing system tablet
I20260809 12:57:02.288717 29929 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:02.288833 29929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:02.292065 29929 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.005s sys 0.000s
I20260809 12:57:02.294126 30046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:02.294553 29929 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:02.294993 29929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 12:57:02.295135 29929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/wal
uuid: "91d3abc7c1254b84bb14a33b01d586d1"
format_stamp: "Formatted at 2026-08-09 12:57:02 on dist-test-slave-gp6n"
server_key: "e3cc8c3bd032d4f02faea397948a3f32"
server_key_iv: "73d00ccc988121278d453cf3ae5744e4"
server_key_version: "encryptionkey@0"
I20260809 12:57:02.300817 29929 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:02.307942 29929 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 12:57:02.308194 29929 log.cc:1196] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 12:57:02.308732 29929 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: Deleting consensus metadata
I20260809 12:57:02.309064 29929 master_runner.cc:315] Copying system tablet from 127.21.235.126:33521
I20260809 12:57:02.312462 29929 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: tablet copy: Beginning tablet copy session from remote peer at address 127.21.235.126:33521
I20260809 12:57:02.333688 30009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.313638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40726 (local address 127.21.235.126:33521)
0809 12:57:02.313836 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:02.313843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:02.318622 (+ 4779us) server_negotiation.cc:407] Connection header received
0809 12:57:02.318851 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:02.318857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:02.318946 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:02.319073 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:02.320315 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.321466 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.323455 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.324085 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.324223 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.326620 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:02.326630 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:02.326637 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:02.326687 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:02.330436 (+ 3749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.331326 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.331333 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.331338 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.331528 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.331987 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.331992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.331995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.332408 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:02.332568 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:02.333065 (+ 497us) server_negotiation.cc:299] Negotiation successful
0809 12:57:02.333314 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:57:02.333724 30053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.313585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.0.0.1:40726)
0809 12:57:02.318480 (+ 4895us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:02.318507 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:02.318683 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:02.319164 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:02.319170 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:02.319806 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:02.320132 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.320145 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.321708 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.321714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.323297 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.323312 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.324260 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.324266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.324440 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.325162 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:02.325190 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:02.326373 (+ 1183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:02.330577 (+ 4204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.330583 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.330588 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.331145 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.331654 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.331659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.331663 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.331842 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.332542 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:02.332547 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:02.332755 (+ 208us) client_negotiation.cc:769] Sending connection context
0809 12:57:02.332917 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 12:57:02.333245 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4684,"thread_start_us":4574,"threads_started":1}
I20260809 12:57:02.335171 29427 tablet_copy_service.cc:140] P e0475d10a6054e859f62c2497daa235c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 91d3abc7c1254b84bb14a33b01d586d1 ({username='oryx', principal='oryx/127.21.235.125@KRBTEST.COM'} at 127.0.0.1:40726)
I20260809 12:57:02.335668 29427 tablet_copy_service.cc:161] P e0475d10a6054e859f62c2497daa235c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 91d3abc7c1254b84bb14a33b01d586d1 at {username='oryx', principal='oryx/127.21.235.125@KRBTEST.COM'} at 127.0.0.1:40726: session id = 91d3abc7c1254b84bb14a33b01d586d1-00000000000000000000000000000000
I20260809 12:57:02.342299 29427 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 12:57:02.345299 29929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 12:57:02.353071 29929 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: tablet copy: Starting download of 0 data blocks...
I20260809 12:57:02.353500 29929 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: tablet copy: Starting download of 1 WAL segments...
I20260809 12:57:02.357686 29929 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 12:57:02.363009 29427 tablet_copy_service.cc:342] P e0475d10a6054e859f62c2497daa235c: Request end of tablet copy session 91d3abc7c1254b84bb14a33b01d586d1-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.235.125@KRBTEST.COM'} at 127.0.0.1:40726
I20260809 12:57:02.363554 29427 tablet_copy_service.cc:433] P e0475d10a6054e859f62c2497daa235c: ending tablet copy session 91d3abc7c1254b84bb14a33b01d586d1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 91d3abc7c1254b84bb14a33b01d586d1
I20260809 12:57:02.380872 29929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:02.383749 30054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:02.387771 30055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:02.390070 30057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:02.392865 29929 server_base.cc:1034] running on GCE node
I20260809 12:57:02.393244 29929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:02.393900 29929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:02.395185 29929 hybrid_clock.cc:648] HybridClock initialized: now 1786280222395131 us; error 27 us; skew 500 ppm
I20260809 12:57:02.395427 29929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:02 dist-test-slave-gp6n krb5kdc[29338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280222, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:02.397300 29929 init.cc:377] Logged in from keytab as oryx/127.21.235.125@KRBTEST.COM (short username oryx)
I20260809 12:57:02.398813 29929 webserver.cc:460] Webserver started at http://127.21.235.125:36203/ using document root <none> and password file <none>
I20260809 12:57:02.399165 29929 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:02.399264 29929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:02.401640 29929 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 12:57:02.403460 30066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:02.403877 29929 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 12:57:02.404038 29929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/wal
uuid: "91d3abc7c1254b84bb14a33b01d586d1"
format_stamp: "Formatted at 2026-08-09 12:57:02 on dist-test-slave-gp6n"
server_key: "e3cc8c3bd032d4f02faea397948a3f32"
server_key_iv: "73d00ccc988121278d453cf3ae5744e4"
server_key_version: "encryptionkey@0"
I20260809 12:57:02.404567 29929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:02.418485 29929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:02.422542 29929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:02.466948 29929 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.125:42291
I20260809 12:57:02.468091 29929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-1/data/info.pb
I20260809 12:57:02.466948 30123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.125:42291 every 8 connection(s)
I20260809 12:57:02.482105 30124 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:57:02.485023 30124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: Bootstrap starting.
I20260809 12:57:02.500525 30124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:57:02.501180 30124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: Bootstrap complete.
I20260809 12:57:02.502571 30124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: NON_VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: true } }
I20260809 12:57:02.502842 30124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 91d3abc7c1254b84bb14a33b01d586d1, State: Initialized, Role: LEARNER
I20260809 12:57:02.503113 30124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: NON_VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: true } }
I20260809 12:57:02.504106 30127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: NON_VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: true } } }
I20260809 12:57:02.504541 30124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:57:02.504612 30127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [sys.catalog]: This master's current role is: LEARNER
I20260809 12:57:02.526665 30138 catalog_manager.cc:1259] Loaded cluster ID: 9ea99b7009ef49d794f2c37b68465cb1
I20260809 12:57:02.526907 30138 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: loading cluster ID for follower catalog manager: success
I20260809 12:57:02.531021 30138 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: acquiring CA information for follower catalog manager: success
I20260809 12:57:02.534013 30138 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 12:57:02.573683 29921 heartbeater.cc:503] Master 127.21.235.126:33521 was elected leader, sending a full tablet report...
W20260809 12:57:02.730895 30043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:02.731294 30043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:02.731472 30043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:02.739810 30043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:57:02.740016 30043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:02.740152 30043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:02.740304 30043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:57:02.740399 30043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:57:02.751804 30043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:44997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:33521,127.21.235.125:42291,127.21.235.124:40403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.124.keytab
--principal=oryx/127.21.235.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.124:40403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.235.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:02.753369 30043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:02.754956 30043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:02.768901 30143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:02.769253 30144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:02.769731 30146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:02.770151 30043 server_base.cc:1034] running on GCE node
I20260809 12:57:02.771134 30043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:02.774044 30043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:02.775345 30043 hybrid_clock.cc:648] HybridClock initialized: now 1786280222775231 us; error 50 us; skew 500 ppm
I20260809 12:57:02.775771 30043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:02 dist-test-slave-gp6n krb5kdc[29338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280222, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:02.779672 30150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.771800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53812 (local address 127.21.235.125:42291)
0809 12:57:02.772543 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:02.772547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:02.772567 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:57:02.772630 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:02.772633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:02.772701 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:02.772818 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:02.773380 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.775007 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.777550 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.778602 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.778764 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.778952 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:02.779247 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 12:57:02.779387 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":466,"threads_started":1}
I20260809 12:57:02.779713 30149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.771568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:42291 (local address 127.0.0.1:53812)
0809 12:57:02.772092 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:02.772111 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:02.772251 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:02.772874 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:02.772881 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:02.772940 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:02.773175 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.773190 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.775147 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.775152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.777384 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.777400 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.778777 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.778784 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.779020 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.779081 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 12:57:02.779179 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 12:57:02.779335 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":288,"threads_started":1}
I20260809 12:57:02.779955 30043 init.cc:377] Logged in from keytab as oryx/127.21.235.124@KRBTEST.COM (short username oryx)
I20260809 12:57:02.781087 30097 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260809 12:57:02.781810 30127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0475d10a6054e859f62c2497daa235c. Latest consensus state: current_term: 1 leader_uuid: "e0475d10a6054e859f62c2497daa235c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: NON_VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: true } } }
I20260809 12:57:02.782105 30127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [sys.catalog]: This master's current role is: LEARNER
I20260809 12:57:02.782169 30043 webserver.cc:460] Webserver started at http://127.21.235.124:40069/ using document root <none> and password file <none>
I20260809 12:57:02.782964 30043 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:02.783134 30043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:02.783455 30043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:02.786690 30043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/data/instance:
uuid: "ede96f6f1f564f3c962f6e4080908730"
format_stamp: "Formatted at 2026-08-09 12:57:02 on dist-test-slave-gp6n"
server_key: "65b3132c209cf608c8ccf6a07d7cd177"
server_key_iv: "76bef2ef3f9adcb3b18190da6b113923"
server_key_version: "encryptionkey@0"
I20260809 12:57:02.787499 30043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/wal/instance:
uuid: "ede96f6f1f564f3c962f6e4080908730"
format_stamp: "Formatted at 2026-08-09 12:57:02 on dist-test-slave-gp6n"
server_key: "65b3132c209cf608c8ccf6a07d7cd177"
server_key_iv: "76bef2ef3f9adcb3b18190da6b113923"
server_key_version: "encryptionkey@0"
I20260809 12:57:02.793279 30043 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:57:02.797246 30156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:02.798761 30043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 12:57:02.798916 30043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/wal
uuid: "ede96f6f1f564f3c962f6e4080908730"
format_stamp: "Formatted at 2026-08-09 12:57:02 on dist-test-slave-gp6n"
server_key: "65b3132c209cf608c8ccf6a07d7cd177"
server_key_iv: "76bef2ef3f9adcb3b18190da6b113923"
server_key_version: "encryptionkey@0"
I20260809 12:57:02.799088 30043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:02.821691 30043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:02.827036 30043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:02.827351 30043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:02.852644 30043 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.124:40403
I20260809 12:57:02.852665 30214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.124:40403 every 8 connection(s)
I20260809 12:57:02.854619 30043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/data/info.pb
I20260809 12:57:02.857709 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 30043
I20260809 12:57:02.859519 30215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:02.868455 30215 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:57:02 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.235.124@KRBTEST.COM
I20260809 12:57:02.889983 30220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.860108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44642 (local address 127.21.235.124:40403)
0809 12:57:02.861005 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:02.861018 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:02.861063 (+ 45us) server_negotiation.cc:407] Connection header received
0809 12:57:02.863089 (+ 2026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:02.863110 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:02.863549 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:02.864006 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:02.865691 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.867239 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.868647 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.869446 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.869650 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.876997 (+ 7347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:02.877010 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:02.877033 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:02.877094 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:02.882808 (+ 5714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.884031 (+ 1223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.884044 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.884056 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.884344 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.884926 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.884933 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.884940 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.885892 (+ 952us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:02.887671 (+ 1779us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:02.888005 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 12:57:02.888415 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":422,"threads_started":1}
Aug 09 12:57:02 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280222, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.124@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM
I20260809 12:57:02.903653 30223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.871563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40732 (local address 127.21.235.126:33521)
0809 12:57:02.886882 (+ 15319us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:02.886888 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:02.888036 (+ 1148us) server_negotiation.cc:407] Connection header received
0809 12:57:02.888098 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:02.888103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:02.888179 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:02.888280 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:02.889931 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.891261 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.893523 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.893996 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.894092 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.897904 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:02.897908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:02.897911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:02.897942 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:02.900901 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.901583 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.901587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.901589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.901715 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.902099 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.902102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.902105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.902432 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:02.902565 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:02.903157 (+ 592us) server_negotiation.cc:299] Negotiation successful
0809 12:57:02.903331 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15174,"thread_start_us":15331,"threads_started":1}
I20260809 12:57:02.903743 30222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.871383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.0.0.1:40732)
0809 12:57:02.887685 (+ 16302us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:02.887730 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:02.887902 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:02.888366 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:02.888373 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:02.889394 (+ 1021us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:02.889748 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.889760 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.891450 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.891456 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.893383 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.893397 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.894194 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.894201 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.894391 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.895099 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:02.895137 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:02.897745 (+ 2608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:02.901014 (+ 3269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.901023 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.901028 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.901479 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.901808 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.901811 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.901813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.901996 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.902580 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:02.902588 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:02.902802 (+ 214us) client_negotiation.cc:769] Sending connection context
0809 12:57:02.903038 (+ 236us) client_negotiation.cc:240] Negotiation successful
0809 12:57:02.903291 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15988,"thread_start_us":15794,"threads_started":1}
I20260809 12:57:02.905194 30215 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:57:02 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280222, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.124@KRBTEST.COM for oryx/127.21.235.125@KRBTEST.COM
I20260809 12:57:02.918884 30222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.905928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:42291 (local address 127.0.0.1:53824)
0809 12:57:02.906191 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:02.906208 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:02.906301 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:02.906644 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:02.906647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:02.906979 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:02.907127 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.907134 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.908238 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.908242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.909393 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.909404 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.910257 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.910261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.910485 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.910983 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:02.910998 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:02.913183 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:02.916579 (+ 3396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.916582 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.916585 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.917019 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.917412 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.917415 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.917417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.917558 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.918045 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:02.918048 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:02.918178 (+ 130us) client_negotiation.cc:769] Sending connection context
0809 12:57:02.918283 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 12:57:02.918600 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260809 12:57:02.919059 30150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.906043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53824 (local address 127.21.235.125:42291)
0809 12:57:02.906258 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:02.906266 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:02.906290 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:57:02.906429 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:02.906434 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:02.906514 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:02.906607 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:02.907278 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.908120 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.909538 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.910113 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.910222 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.913448 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:02.913453 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:02.913456 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:02.913489 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:02.916459 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.917127 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.917131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.917134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.917317 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.917665 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.917671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.917673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.917949 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:02.918070 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:02.918557 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 12:57:02.918745 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 12:57:02.920086 30215 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:57:02 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280222, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.124@KRBTEST.COM for oryx/127.21.235.124@KRBTEST.COM
I20260809 12:57:02.933537 30222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.920819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:40403 (local address 127.0.0.1:44652)
0809 12:57:02.921018 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:02.921036 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:02.921127 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:02.921466 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:02.921469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:02.921810 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:02.921968 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.921976 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.923188 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.923192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.924119 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.924130 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.924807 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.924811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.924981 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.925549 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:02.925565 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:02.927734 (+ 2169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:02.931114 (+ 3380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.931118 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.931121 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.931623 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.931996 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.931999 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.932006 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.932187 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.932705 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:02.932708 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:02.932868 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 12:57:02.932978 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 12:57:02.933192 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 12:57:02.933728 30220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.920894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44652 (local address 127.21.235.124:40403)
0809 12:57:02.921051 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:02.921055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:02.921086 (+ 31us) server_negotiation.cc:407] Connection header received
0809 12:57:02.921261 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:02.921267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:02.921331 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:02.921403 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:02.922084 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.923068 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.924231 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.924664 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.924752 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.927939 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:02.927944 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:02.927948 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:02.927977 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:02.930947 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.931749 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.931753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.931755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.931890 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.932281 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.932284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.932286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.932601 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:02.932714 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:02.933064 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 12:57:02.933361 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":49}
I20260809 12:57:02.939433 30215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: Bootstrap starting.
I20260809 12:57:02.943131 30215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:02.944474 30215 log.cc:824] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: Log is configured to *not* fsync() on all Append() calls
I20260809 12:57:02.947621 30215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: No bootstrap required, opened a new log
I20260809 12:57:02.955433 30215 raft_consensus.cc:348] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } }
I20260809 12:57:02.955758 30215 raft_consensus.cc:374] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:02.955899 30215 raft_consensus.cc:729] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ede96f6f1f564f3c962f6e4080908730, State: Initialized, Role: FOLLOWER
I20260809 12:57:02.956435 30215 consensus_queue.cc:260] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } }
I20260809 12:57:02.957427 30225 sys_catalog.cc:455] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } } }
I20260809 12:57:02.957775 30225 sys_catalog.cc:458] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:57:02.958338 30215 sys_catalog.cc:564] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 12:57:02.969688 30237 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 12:57:02.969880 30237 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 12:57:02.974098 30043 master_runner.cc:428] Detected that this master ede96f6f1f564f3c962f6e4080908730 is joining an existing cluster
I20260809 12:57:02.974239 30043 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.235.124:40403
I20260809 12:57:02.997464 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 29474
I20260809 12:57:03.000787 30223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.978891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40738 (local address 127.21.235.126:33521)
0809 12:57:02.979084 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:02.979090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:02.980172 (+ 1082us) server_negotiation.cc:407] Connection header received
0809 12:57:02.980446 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:02.980452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:02.980555 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:02.980632 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:02.982358 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.983570 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.985858 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.986556 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.986659 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.989752 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:02.989759 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:02.989764 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:02.989806 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:02.993704 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.994610 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.994616 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.994620 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.994781 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:02.995324 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:02.995330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:02.995333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:02.996235 (+ 902us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:02.996391 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:03.000142 (+ 3751us) server_negotiation.cc:299] Negotiation successful
0809 12:57:03.000394 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:57:03.001876 30243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.979014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.0.0.1:40738)
0809 12:57:02.980059 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:02.980082 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:02.980282 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:02.980857 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:02.980861 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:02.981774 (+ 913us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:02.982176 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.982189 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.983793 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.983798 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.985661 (+ 1863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.985676 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.987102 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.987110 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.987361 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:02.988193 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:02.988217 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:02.989494 (+ 1277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:02.993897 (+ 4403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.993902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.993906 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.994454 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.994946 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:02.994950 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:02.994954 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:02.995161 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:02.996505 (+ 1344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:02.996514 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:02.998116 (+ 1602us) client_negotiation.cc:769] Sending connection context
0809 12:57:03.001096 (+ 2980us) client_negotiation.cc:240] Negotiation successful
0809 12:57:03.001419 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"thread_start_us":726,"threads_started":1}
I20260809 12:57:03.002588 30225 raft_consensus.cc:482] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:57:03.002893 30225 raft_consensus.cc:504] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } }
I20260809 12:57:03.004611 30225 leader_election.cc:288] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0475d10a6054e859f62c2497daa235c (127.21.235.126:33521), 91d3abc7c1254b84bb14a33b01d586d1 (127.21.235.125:42291)
I20260809 12:57:03.005503 29415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ede96f6f1f564f3c962f6e4080908730" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0475d10a6054e859f62c2497daa235c" is_pre_election: true
I20260809 12:57:03.010728 30097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ede96f6f1f564f3c962f6e4080908730" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91d3abc7c1254b84bb14a33b01d586d1" is_pre_election: true
I20260809 12:57:03.011685 30160 leader_election.cc:302] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ede96f6f1f564f3c962f6e4080908730; no voters: 91d3abc7c1254b84bb14a33b01d586d1, e0475d10a6054e859f62c2497daa235c
I20260809 12:57:03.012475 30225 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 12:57:03.016041 30220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.978927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44668 (local address 127.21.235.124:40403)
0809 12:57:02.979294 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:02.979300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:02.990430 (+ 11130us) server_negotiation.cc:407] Connection header received
0809 12:57:02.990885 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:02.990893 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:02.990976 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:02.991055 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:02.992969 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.994566 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.998490 (+ 3924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.999712 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.999958 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:03.008516 (+ 8558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:03.008523 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:03.008530 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:03.008565 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:03.012337 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:03.013549 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:03.013554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:03.013557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:03.013708 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:03.014171 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:03.014176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:03.014179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:03.014721 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:03.014867 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:03.015432 (+ 565us) server_negotiation.cc:299] Negotiation successful
0809 12:57:03.015641 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":160,"spinlock_wait_cycles":26496}
I20260809 12:57:03.018080 30245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.979862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:40403 (local address 127.0.0.1:44668)
0809 12:57:02.990280 (+ 10418us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:02.990310 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:02.990721 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:02.991303 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:02.991311 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:02.992331 (+ 1020us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:02.992806 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.992819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:02.995189 (+ 2370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:02.995198 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:02.998283 (+ 3085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:02.998299 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.000005 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.000013 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:03.000444 (+ 431us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:03.005498 (+ 5054us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:03.005524 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:03.008332 (+ 2808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:03.012912 (+ 4580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:03.012917 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:03.012920 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:03.013389 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:03.013850 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:03.013856 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:03.013861 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:03.014035 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:03.014963 (+ 928us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:03.014970 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:03.015138 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 12:57:03.017186 (+ 2048us) client_negotiation.cc:240] Negotiation successful
0809 12:57:03.017528 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10199,"mutex_wait_us":224,"thread_start_us":15910,"threads_started":1}
I20260809 12:57:03.023171 30150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.980923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53826 (local address 127.21.235.125:42291)
0809 12:57:02.981154 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:02.981163 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:02.996868 (+ 15705us) server_negotiation.cc:407] Connection header received
0809 12:57:02.997152 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:02.997158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:02.997243 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:02.997323 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:03.000811 (+ 3488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.001903 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.008119 (+ 6216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.008863 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.008999 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:03.015581 (+ 6582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:03.015589 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:03.015595 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:03.015634 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:03.019369 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:03.020300 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:03.020306 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:03.020310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:03.020480 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:03.020930 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:03.020937 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:03.020943 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:03.021318 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:03.021447 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:03.022082 (+ 635us) server_negotiation.cc:299] Negotiation successful
0809 12:57:03.022315 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 12:57:03.024190 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.980274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:42291 (local address 127.0.0.1:53826)
0809 12:57:02.996760 (+ 16486us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:02.996780 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:02.996991 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:02.997685 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:02.997690 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:02.999526 (+ 1836us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:03.000073 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:03.000085 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.006560 (+ 6475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.006565 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:03.007945 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:03.007958 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.009450 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.009457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:03.013593 (+ 4136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:03.014176 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:03.014198 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:03.015348 (+ 1150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:03.019560 (+ 4212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:03.019564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:03.019568 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:03.020153 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:03.020621 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:03.020626 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:03.020629 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:03.020798 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:03.021566 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:03.021571 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:03.021784 (+ 213us) client_negotiation.cc:769] Sending connection context
0809 12:57:03.023425 (+ 1641us) client_negotiation.cc:240] Negotiation successful
0809 12:57:03.023762 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16263,"thread_start_us":10170,"threads_started":1}
Aug 09 12:57:03 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280219, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.126@KRBTEST.COM for oryx/127.21.235.124@KRBTEST.COM
Aug 09 12:57:03 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.235.65@KRBTEST.COM
I20260809 12:57:03.033903 30220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:03.014629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44680 (local address 127.21.235.124:40403)
0809 12:57:03.016669 (+ 2040us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:03.016676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:03.016692 (+ 16us) server_negotiation.cc:407] Connection header received
0809 12:57:03.016778 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:03.016783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:03.016894 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:03.017005 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:03.018003 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.019377 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.021208 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.021968 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.022643 (+ 675us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:03.026550 (+ 3907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:03.026561 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:03.026566 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:03.026606 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:03.030649 (+ 4043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:03.031602 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:03.031611 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:03.031621 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:03.031771 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:03.032241 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:03.032246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:03.032250 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:03.032615 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:03.032777 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:03.033290 (+ 513us) server_negotiation.cc:299] Negotiation successful
0809 12:57:03.033530 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1931,"thread_start_us":4385,"threads_started":1}
I20260809 12:57:03.034119 30149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:03.013696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:40403 (local address 127.0.0.1:44680)
0809 12:57:03.014194 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:03.014218 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:03.014366 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:03.017045 (+ 2679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:03.017052 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:03.017589 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:03.017781 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:03.017794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.019525 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.019531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:03.021038 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:03.021053 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.022134 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.022140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:03.022317 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:03.022930 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:03.022952 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:03.026304 (+ 3352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:03.030830 (+ 4526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:03.030836 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:03.030840 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:03.031460 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:03.031905 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:03.031912 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:03.031918 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:03.032095 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:03.032758 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:03.032763 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:03.032981 (+ 218us) client_negotiation.cc:769] Sending connection context
0809 12:57:03.033159 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 12:57:03.033739 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325}
I20260809 12:57:03.036078 29403 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.235.124:40403: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ede96f6f1f564f3c962f6e4080908730"
member_type: NON_VOTER
last_known_addr {
host: "127.21.235.124"
port: 40403
}
attrs {
promote: true
}
}
dest_uuid: "e0475d10a6054e859f62c2497daa235c"
cas_config_opid_index: 5
I20260809 12:57:03.037273 29403 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: NON_VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: true } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: NON_VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: true } }
I20260809 12:57:03.038947 30148 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ede96f6f1f564f3c962f6e4080908730 (127.21.235.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: NON_VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: true } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: NON_VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: true } } }
I20260809 12:57:03.039639 30187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:03.040625 30246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:02.999256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53778 (local address 127.21.235.65:39115)
0809 12:57:03.016094 (+ 16838us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:03.016107 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:03.016149 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:57:03.016247 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:03.016259 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:03.016616 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:03.020470 (+ 3854us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:03.020775 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.022817 (+ 2042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.024799 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.025809 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.026010 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:03.032308 (+ 6298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:03.032320 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:03.032335 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:03.032369 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:03.036958 (+ 4589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:03.037957 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:03.037969 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:03.037975 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:03.038156 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:03.038702 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:03.038709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:03.038712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:03.039158 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:03.039544 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:03.039858 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 12:57:03.040171 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16596,"thread_start_us":17551,"threads_started":1}
I20260809 12:57:03.042047 29614 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:03.043371 30187 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 1 FOLLOWER]: Refusing update from remote peer e0475d10a6054e859f62c2497daa235c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 12:57:03.047712 30097 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 1 LEARNER]: Refusing update from remote peer e0475d10a6054e859f62c2497daa235c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 12:57:03.053417 30148 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.235.124:40403
I20260809 12:57:03.053610 30250 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: NON_VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 12:57:03.054177 30043 master.cc:537] Master@127.21.235.124:40403 shutting down...
I20260809 12:57:03.055245 30248 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [LEADER]: Connected to new peer: Peer: permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: NON_VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 12:57:03.059937 30251 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e0475d10a6054e859f62c2497daa235c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: NON_VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: true } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: NON_VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: true } } }
I20260809 12:57:03.060444 30251 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:03.080144 30097 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ede96f6f1f564f3c962f6e4080908730 (127.21.235.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: NON_VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: true } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: NON_VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: true } } }
I20260809 12:57:03.085453 30127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0475d10a6054e859f62c2497daa235c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: NON_VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: true } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: NON_VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: true } } }
I20260809 12:57:03.085836 30127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [sys.catalog]: This master's current role is: LEARNER
I20260809 12:57:03.087253 30148 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c: attempting to promote NON_VOTER 91d3abc7c1254b84bb14a33b01d586d1 to VOTER
W20260809 12:57:03.087424 29383 consensus_peers.cc:487] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c -> Peer ede96f6f1f564f3c962f6e4080908730 (127.21.235.124:40403): Couldn't send request to peer ede96f6f1f564f3c962f6e4080908730. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 12:57:03.088354 30148 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: NON_VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: true } }
I20260809 12:57:03.090981 30097 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 1 LEARNER]: Refusing update from remote peer e0475d10a6054e859f62c2497daa235c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260809 12:57:03.091352 30163 tablet.cc:2369] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:57:03.091794 30251 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [LEADER]: Connected to new peer: Peer: permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 12:57:03.093291 30043 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:57:03.093681 30043 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:03.093863 30043 tablet_replica.cc:330] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: stopping tablet replica
I20260809 12:57:03.094650 30248 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 91d3abc7c1254b84bb14a33b01d586d1 (127.21.235.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: NON_VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: true } } }
I20260809 12:57:03.095443 30097 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 91d3abc7c1254b84bb14a33b01d586d1 (127.21.235.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: NON_VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: true } } }
I20260809 12:57:03.099066 30148 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e0475d10a6054e859f62c2497daa235c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: NON_VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: true } } }
I20260809 12:57:03.099346 30148 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:03.099709 30148 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0475d10a6054e859f62c2497daa235c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: NON_VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: true } } }
I20260809 12:57:03.099947 30148 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: This master's current role is: LEADER
W20260809 12:57:03.100409 29383 consensus_peers.cc:487] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c -> Peer ede96f6f1f564f3c962f6e4080908730 (127.21.235.124:40403): Couldn't send request to peer ede96f6f1f564f3c962f6e4080908730. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260809 12:57:03.100324 30127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0475d10a6054e859f62c2497daa235c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: NON_VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: true } } }
I20260809 12:57:03.100633 30127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:57:03.100845 30043 master.cc:552] Master@127.21.235.124:40403 shutdown complete.
I20260809 12:57:03.100934 30043 master_runner.cc:305] Clearing existing system tablet
I20260809 12:57:03.101312 30043 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:03.101428 30043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:03.105321 30043 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:57:03.109115 30257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:03.109486 30043 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:03.109920 30043 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260809 12:57:03.110056 30043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/wal
uuid: "ede96f6f1f564f3c962f6e4080908730"
format_stamp: "Formatted at 2026-08-09 12:57:02 on dist-test-slave-gp6n"
server_key: "65b3132c209cf608c8ccf6a07d7cd177"
server_key_iv: "76bef2ef3f9adcb3b18190da6b113923"
server_key_version: "encryptionkey@0"
I20260809 12:57:03.115295 30043 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:03.122299 30043 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 12:57:03.122555 30043 log.cc:1196] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 12:57:03.123126 30043 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: Deleting consensus metadata
I20260809 12:57:03.123417 30043 master_runner.cc:315] Copying system tablet from 127.21.235.126:33521
I20260809 12:57:03.126309 30043 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: tablet copy: Beginning tablet copy session from remote peer at address 127.21.235.126:33521
I20260809 12:57:03.143678 30223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:03.127501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40746 (local address 127.21.235.126:33521)
0809 12:57:03.127755 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:03.127764 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:03.128497 (+ 733us) server_negotiation.cc:407] Connection header received
0809 12:57:03.128724 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:03.128730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:03.128830 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:03.128958 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:03.130192 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.131412 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.133389 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.134045 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.134176 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:03.136670 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:03.136677 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:03.136682 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:03.136727 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:03.140609 (+ 3882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:03.141371 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:03.141378 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:03.141391 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:03.141558 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:03.141985 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:03.141990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:03.141994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:03.142356 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:03.142556 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:03.143053 (+ 497us) server_negotiation.cc:299] Negotiation successful
0809 12:57:03.143278 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260809 12:57:03.143720 30264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:03.127500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.0.0.1:40746)
0809 12:57:03.128350 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:03.128370 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:03.128543 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:03.129007 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:03.129012 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:03.129607 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:03.130025 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:03.130037 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.131604 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.131610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:03.133218 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:03.133232 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.134213 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.134219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:03.134539 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:03.135304 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:03.135327 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:03.136477 (+ 1150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:03.140748 (+ 4271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:03.140753 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:03.140758 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:03.141248 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:03.141684 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:03.141689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:03.141693 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:03.141865 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:03.142579 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:03.142584 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:03.142771 (+ 187us) client_negotiation.cc:769] Sending connection context
0809 12:57:03.142919 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 12:57:03.143258 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":388,"threads_started":1}
I20260809 12:57:03.144606 29427 tablet_copy_service.cc:140] P e0475d10a6054e859f62c2497daa235c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ede96f6f1f564f3c962f6e4080908730 ({username='oryx', principal='oryx/127.21.235.124@KRBTEST.COM'} at 127.0.0.1:40746)
I20260809 12:57:03.144783 29427 tablet_copy_service.cc:161] P e0475d10a6054e859f62c2497daa235c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ede96f6f1f564f3c962f6e4080908730 at {username='oryx', principal='oryx/127.21.235.124@KRBTEST.COM'} at 127.0.0.1:40746: session id = ede96f6f1f564f3c962f6e4080908730-00000000000000000000000000000000
I20260809 12:57:03.148552 29427 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 12:57:03.150569 30043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 12:57:03.157724 30043 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: tablet copy: Starting download of 0 data blocks...
I20260809 12:57:03.158028 30043 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: tablet copy: Starting download of 1 WAL segments...
I20260809 12:57:03.160569 30043 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 12:57:03.164978 29427 tablet_copy_service.cc:342] P e0475d10a6054e859f62c2497daa235c: Request end of tablet copy session ede96f6f1f564f3c962f6e4080908730-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.235.124@KRBTEST.COM'} at 127.0.0.1:40746
I20260809 12:57:03.165104 29427 tablet_copy_service.cc:433] P e0475d10a6054e859f62c2497daa235c: ending tablet copy session ede96f6f1f564f3c962f6e4080908730-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ede96f6f1f564f3c962f6e4080908730
I20260809 12:57:03.178515 30043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 12:57:03.179447 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 29474
W20260809 12:57:03.183909 30266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:03.193164 30267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:03.199939 30270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:03.206274 30043 server_base.cc:1034] running on GCE node
I20260809 12:57:03.206785 30043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:03.207552 30043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:03.208840 30043 hybrid_clock.cc:648] HybridClock initialized: now 1786280223208594 us; error 235 us; skew 500 ppm
I20260809 12:57:03.209184 30043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:03 dist-test-slave-gp6n krb5kdc[29338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280223, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:03.211700 30043 init.cc:377] Logged in from keytab as oryx/127.21.235.124@KRBTEST.COM (short username oryx)
I20260809 12:57:03.212690 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:39115
--local_ip_for_outbound_sockets=127.21.235.65
--tserver_master_addrs=127.21.235.126:33521
--webserver_port=38813
--webserver_interface=127.21.235.65
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.65.keytab
--principal=oryx/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:44997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.235.126:33521,127.21.235.125:42291
--tserver_master_addrs=127.21.235.126:33521,127.21.235.125:42291,127.21.235.124:40403 with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:57:03.218461 30043 webserver.cc:460] Webserver started at http://127.21.235.124:40515/ using document root <none> and password file <none>
I20260809 12:57:03.218843 30043 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:03.218972 30043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:03.222508 30043 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 12:57:03.224459 30278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:03.224917 30043 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260809 12:57:03.225044 30043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/wal
uuid: "ede96f6f1f564f3c962f6e4080908730"
format_stamp: "Formatted at 2026-08-09 12:57:02 on dist-test-slave-gp6n"
server_key: "65b3132c209cf608c8ccf6a07d7cd177"
server_key_iv: "76bef2ef3f9adcb3b18190da6b113923"
server_key_version: "encryptionkey@0"
I20260809 12:57:03.225239 30043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:03.256222 30043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:03.261145 30043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:03.288542 30043 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.124:40403
I20260809 12:57:03.288617 30334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.124:40403 every 8 connection(s)
I20260809 12:57:03.289860 30043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/master-2/data/info.pb
I20260809 12:57:03.296018 30335 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:57:03.300729 30335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: Bootstrap starting.
I20260809 12:57:03.320305 30335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:57:03.320924 30335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: Bootstrap complete.
I20260809 12:57:03.322146 30335 raft_consensus.cc:348] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: NON_VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: true } }
I20260809 12:57:03.322499 30335 raft_consensus.cc:729] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ede96f6f1f564f3c962f6e4080908730, State: Initialized, Role: LEARNER
I20260809 12:57:03.322788 30335 consensus_queue.cc:260] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: NON_VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: true } }
I20260809 12:57:03.324156 30335 sys_catalog.cc:564] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:57:03.323830 30338 sys_catalog.cc:455] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: NON_VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: true } } }
I20260809 12:57:03.324497 30338 sys_catalog.cc:458] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [sys.catalog]: This master's current role is: LEARNER
I20260809 12:57:03.335918 30349 catalog_manager.cc:1259] Loaded cluster ID: 9ea99b7009ef49d794f2c37b68465cb1
I20260809 12:57:03.336035 30349 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: loading cluster ID for follower catalog manager: success
I20260809 12:57:03.340205 30349 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: acquiring CA information for follower catalog manager: success
I20260809 12:57:03.342846 30349 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 12:57:03.617085 30277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:03.617408 30277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:03.617576 30277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:03.625653 30277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:03.625797 30277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:03.626014 30277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:57:03.637394 30277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:44997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.65.keytab
--principal=oryx/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:39115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=38813
--tserver_master_addrs=127.21.235.126:33521,127.21.235.125:42291,127.21.235.124:40403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:03.638904 30277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:03.640499 30277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:03.655049 30355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:03.655157 30356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:03.655079 30358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:03.657730 30277 server_base.cc:1034] running on GCE node
I20260809 12:57:03.658679 30277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:03.659917 30277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:03.661158 30277 hybrid_clock.cc:648] HybridClock initialized: now 1786280223661068 us; error 79 us; skew 500 ppm
I20260809 12:57:03.661572 30277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:57:03.664397 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:03.656278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:40403 (local address 127.0.0.1:44684)
0809 12:57:03.658718 (+ 2440us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:03.658733 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:03.658900 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:03.659312 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:03.659315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:03.659335 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:03.659528 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:03.659536 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.660915 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.660919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:03.662219 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:03.662233 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.663824 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.663829 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:03.663969 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:03.663995 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 12:57:03.664054 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 12:57:03.664159 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2347,"thread_start_us":2272,"threads_started":1}
I20260809 12:57:03.664649 30361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:03.656704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44684 (local address 127.21.235.124:40403)
0809 12:57:03.657471 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:03.657481 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:03.658848 (+ 1367us) server_negotiation.cc:407] Connection header received
0809 12:57:03.659055 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:03.659060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:03.659160 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:03.659282 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:03.659662 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.660756 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.662382 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.663638 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.663829 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:03.664024 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:03.664104 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 12:57:03.664245 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":331,"threads_started":1}
Aug 09 12:57:03 dist-test-slave-gp6n krb5kdc[29338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280223, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:03.665685 30308 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 12:57:03.666107 30277 init.cc:377] Logged in from keytab as oryx/127.21.235.65@KRBTEST.COM (short username oryx)
I20260809 12:57:03.666359 30338 sys_catalog.cc:455] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0475d10a6054e859f62c2497daa235c. Latest consensus state: current_term: 1 leader_uuid: "e0475d10a6054e859f62c2497daa235c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: NON_VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: true } } }
I20260809 12:57:03.666880 30338 sys_catalog.cc:458] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [sys.catalog]: This master's current role is: LEARNER
I20260809 12:57:03.667944 30277 webserver.cc:460] Webserver started at http://127.21.235.65:38813/ using document root <none> and password file <none>
I20260809 12:57:03.668695 30277 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:03.668850 30277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:03.674584 30277 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 12:57:03.677719 30369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:03.679342 30277 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:57:03.679512 30277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "e2e31488363744cf90ae9aac7c1883d9"
format_stamp: "Formatted at 2026-08-09 12:57:00 on dist-test-slave-gp6n"
server_key: "e3fde278a7289fd250d4f4c200afdc78"
server_key_iv: "24ea9a2399d91a076c9fc3c55322cab9"
server_key_version: "encryptionkey@0"
I20260809 12:57:03.680682 30277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:03.698536 30277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:03.702447 30277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:03.702823 30277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:03.704172 30277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:03.705901 30277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:03.706032 30277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:03.706143 30277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:03.706219 30277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:03.775539 30277 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:39115
I20260809 12:57:03.775543 30493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:39115 every 8 connection(s)
I20260809 12:57:03.777539 30277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:57:03.786375 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 30277
I20260809 12:57:03.787225 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 29628
Aug 09 12:57:03 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280223, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.124@KRBTEST.COM
Aug 09 12:57:03 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280223, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM
Aug 09 12:57:03 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280223, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.125@KRBTEST.COM
I20260809 12:57:03.817226 30361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:03.781944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:51273 (local address 127.21.235.124:40403)
0809 12:57:03.782138 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:03.782143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:03.783823 (+ 1680us) server_negotiation.cc:407] Connection header received
0809 12:57:03.788089 (+ 4266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:03.788098 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:03.788211 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:03.788300 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:03.793697 (+ 5397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.795096 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.799575 (+ 4479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.800238 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.800373 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:03.807483 (+ 7110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:03.807491 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:03.807497 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:03.807547 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:03.812362 (+ 4815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:03.813377 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:03.813386 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:03.813391 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:03.813575 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:03.814346 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:03.814351 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:03.814354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:03.814754 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:03.814910 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:03.816573 (+ 1663us) server_negotiation.cc:299] Negotiation successful
0809 12:57:03.816842 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 12:57:03 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.235.66@KRBTEST.COM
I20260809 12:57:03.819299 30500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:03.782987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:40403 (local address 127.21.235.65:51273)
0809 12:57:03.783722 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:03.783736 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:03.787900 (+ 4164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:03.788900 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:03.788918 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:03.790208 (+ 1290us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:03.791891 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:03.791928 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.795345 (+ 3417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.795350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:03.799419 (+ 4069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:03.799434 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.800675 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.800692 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:03.801172 (+ 480us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:03.802042 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:03.802084 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:03.807206 (+ 5122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:03.812520 (+ 5314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:03.812530 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:03.812548 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:03.813232 (+ 684us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:03.813769 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:03.813774 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:03.813777 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:03.814177 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:03.815029 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:03.815039 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:03.815518 (+ 479us) client_negotiation.cc:769] Sending connection context
0809 12:57:03.817503 (+ 1985us) client_negotiation.cc:240] Negotiation successful
0809 12:57:03.817784 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"mutex_wait_us":3,"spinlock_wait_cycles":89344,"thread_start_us":1687,"threads_started":1}
I20260809 12:57:03.819450 30499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:03.782338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:59243 (local address 127.21.235.126:33521)
0809 12:57:03.783013 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:03.783018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:03.783540 (+ 522us) server_negotiation.cc:407] Connection header received
0809 12:57:03.787822 (+ 4282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:03.787829 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:03.787936 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:03.788057 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:03.793655 (+ 5598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.795205 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.800428 (+ 5223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.801296 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.801425 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:03.807564 (+ 6139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:03.807571 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:03.807577 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:03.807624 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:03.812463 (+ 4839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:03.814000 (+ 1537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:03.814006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:03.814009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:03.814158 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:03.814842 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:03.814850 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:03.814853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:03.815251 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:03.815409 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:03.818789 (+ 3380us) server_negotiation.cc:299] Negotiation successful
0809 12:57:03.819043 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":859,"threads_started":1}
I20260809 12:57:03.819830 30498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:03.781635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.65:59243)
0809 12:57:03.783316 (+ 1681us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:03.783383 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:03.787433 (+ 4050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:03.792255 (+ 4822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:03.792264 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:03.792956 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:03.793445 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:03.793463 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.797682 (+ 4219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.797688 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:03.800271 (+ 2583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:03.800287 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.801670 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.801676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:03.802485 (+ 809us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:03.803138 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:03.803162 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:03.807298 (+ 4136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:03.812631 (+ 5333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:03.812636 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:03.812640 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:03.813829 (+ 1189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:03.814299 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:03.814305 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:03.814311 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:03.814532 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:03.815477 (+ 945us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:03.815483 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:03.815681 (+ 198us) client_negotiation.cc:769] Sending connection context
0809 12:57:03.816394 (+ 713us) client_negotiation.cc:240] Negotiation successful
0809 12:57:03.817017 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1284,"mutex_wait_us":561,"spinlock_wait_cycles":1043328,"thread_start_us":1553,"threads_started":1}
I20260809 12:57:03.821215 30501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:03.784416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:42291 (local address 127.21.235.65:45635)
0809 12:57:03.794636 (+ 10220us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:03.794656 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:03.794761 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:03.796024 (+ 1263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:03.796031 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:03.797344 (+ 1313us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:03.797854 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:03.797870 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.801832 (+ 3962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.801838 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:03.804379 (+ 2541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:03.804396 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.806310 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.806319 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:03.806556 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:03.807213 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:03.807233 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:03.812228 (+ 4995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:03.817271 (+ 5043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:03.817277 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:03.817282 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:03.818205 (+ 923us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:03.818722 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:03.818727 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:03.818730 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:03.819377 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:03.820167 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:03.820171 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:03.820335 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 12:57:03.820497 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 12:57:03.820764 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10095,"mutex_wait_us":6,"thread_start_us":10430,"threads_started":1}
I20260809 12:57:03.821609 30495 heartbeater.cc:347] Connected to a master server at 127.21.235.126:33521
I20260809 12:57:03.821672 30494 heartbeater.cc:347] Connected to a master server at 127.21.235.124:40403
I20260809 12:57:03.821962 30495 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:03.822201 30502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:03.784645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:45635 (local address 127.21.235.125:42291)
0809 12:57:03.795429 (+ 10784us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:03.795437 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:03.795469 (+ 32us) server_negotiation.cc:407] Connection header received
0809 12:57:03.795564 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:03.795570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:03.795729 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:03.795962 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:03.798093 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.801652 (+ 3559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.804595 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.805980 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.806175 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:03.812492 (+ 6317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:03.812499 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:03.812503 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:03.812547 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:03.817108 (+ 4561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:03.818374 (+ 1266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:03.818379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:03.818382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:03.818583 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:03.819562 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:03.819568 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:03.819571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:03.819958 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:03.820093 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:03.821506 (+ 1413us) server_negotiation.cc:299] Negotiation successful
0809 12:57:03.821761 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10634,"thread_start_us":10703,"threads_started":1}
I20260809 12:57:03.821975 30494 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:03.823199 30494 heartbeater.cc:511] Master 127.21.235.124:40403 requested a full tablet report, sending...
I20260809 12:57:03.823563 30495 heartbeater.cc:511] Master 127.21.235.126:33521 requested a full tablet report, sending...
I20260809 12:57:03.824113 30496 heartbeater.cc:347] Connected to a master server at 127.21.235.125:42291
I20260809 12:57:03.824244 30496 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:03.824756 30496 heartbeater.cc:511] Master 127.21.235.125:42291 requested a full tablet report, sending...
I20260809 12:57:03.825877 29402 ts_manager.cc:194] Re-registered known tserver with Master: e2e31488363744cf90ae9aac7c1883d9 (127.21.235.65:39115)
I20260809 12:57:03.826208 30298 ts_manager.cc:194] Registered new tserver with Master: e2e31488363744cf90ae9aac7c1883d9 (127.21.235.65:39115)
I20260809 12:57:03.826725 30085 ts_manager.cc:194] Registered new tserver with Master: e2e31488363744cf90ae9aac7c1883d9 (127.21.235.65:39115)
I20260809 12:57:03.827484 29402 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:59243
I20260809 12:57:03.830180 30504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:03.789078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37250 (local address 127.21.235.66:40791)
0809 12:57:03.801530 (+ 12452us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:03.801541 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:03.801575 (+ 34us) server_negotiation.cc:407] Connection header received
0809 12:57:03.801645 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:03.801655 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:03.801963 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:03.802141 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:03.805125 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.807570 (+ 2445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.811463 (+ 3893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:03.812473 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:03.812594 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:03.820936 (+ 8342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:03.820950 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:03.820967 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:03.821020 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:03.826751 (+ 5731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:03.827730 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:03.827736 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:03.827738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:03.827908 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:03.828475 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:03.828487 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:03.828490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:03.828929 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:03.829247 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:03.829510 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 12:57:03.829787 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12254,"thread_start_us":12063,"threads_started":1}
I20260809 12:57:03.831437 29763 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:03.931107 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 29628
I20260809 12:57:03.951087 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:40791
--local_ip_for_outbound_sockets=127.21.235.66
--tserver_master_addrs=127.21.235.126:33521
--webserver_port=46397
--webserver_interface=127.21.235.66
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.66.keytab
--principal=oryx/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:44997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.235.126:33521,127.21.235.125:42291
--tserver_master_addrs=127.21.235.126:33521,127.21.235.125:42291,127.21.235.124:40403 with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:57:04.267910 30248 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0475d10a6054e859f62c2497daa235c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: NON_VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: true } } }
I20260809 12:57:04.268379 30248 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:04.269711 30248 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c: attempting to promote NON_VOTER ede96f6f1f564f3c962f6e4080908730 to VOTER
I20260809 12:57:04.270306 30248 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false } }
I20260809 12:57:04.272680 30308 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 1 LEARNER]: Refusing update from remote peer e0475d10a6054e859f62c2497daa235c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 12:57:04.272678 30097 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 1 FOLLOWER]: Refusing update from remote peer e0475d10a6054e859f62c2497daa235c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 12:57:04.273307 30508 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 12:57:04.273754 30248 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [LEADER]: Connected to new peer: Peer: permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 12:57:04.277026 30508 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ede96f6f1f564f3c962f6e4080908730 (127.21.235.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false } } }
I20260809 12:57:04.277652 30097 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ede96f6f1f564f3c962f6e4080908730 (127.21.235.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false } } }
I20260809 12:57:04.279016 30308 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ede96f6f1f564f3c962f6e4080908730 (127.21.235.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false } } }
I20260809 12:57:04.285507 30248 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e0475d10a6054e859f62c2497daa235c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false } } }
I20260809 12:57:04.285918 30248 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:04.286442 30248 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0475d10a6054e859f62c2497daa235c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false } } }
I20260809 12:57:04.286792 30248 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:04.287195 30248 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0475d10a6054e859f62c2497daa235c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false } } }
I20260809 12:57:04.287499 30248 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:04.288403 30513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0475d10a6054e859f62c2497daa235c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false } } }
I20260809 12:57:04.288890 30513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:57:04.289780 30514 sys_catalog.cc:455] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0475d10a6054e859f62c2497daa235c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false } } }
I20260809 12:57:04.290246 30514 sys_catalog.cc:458] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 12:57:04.309394 30506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:04.309693 30506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:04.309789 30506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:04.317215 30506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:04.317330 30506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:04.317453 30506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:57:04.328136 30506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:44997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.66.keytab
--principal=oryx/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:40791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=46397
--tserver_master_addrs=127.21.235.126:33521,127.21.235.125:42291,127.21.235.124:40403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:04.329435 30506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:04.330828 30506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:04.344838 30521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:04.344841 30522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:04.345404 30524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:04.346463 30506 server_base.cc:1034] running on GCE node
I20260809 12:57:04.347174 30506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:04.348212 30506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:04.349483 30506 hybrid_clock.cc:648] HybridClock initialized: now 1786280224349426 us; error 62 us; skew 500 ppm
I20260809 12:57:04.349872 30506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:04 dist-test-slave-gp6n krb5kdc[29338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280224, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:04.353470 30506 init.cc:377] Logged in from keytab as oryx/127.21.235.66@KRBTEST.COM (short username oryx)
I20260809 12:57:04.355028 30506 webserver.cc:460] Webserver started at http://127.21.235.66:46397/ using document root <none> and password file <none>
I20260809 12:57:04.355733 30506 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:04.355873 30506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:04.361801 30506 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 12:57:04.364753 30531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:04.366111 30506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:57:04.366240 30506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "86e19c19da6246d89330b6a6f23b0c50"
format_stamp: "Formatted at 2026-08-09 12:57:00 on dist-test-slave-gp6n"
server_key: "af4d3583cb135d0be464dc9ce1cb5a18"
server_key_iv: "0f85efbd76d5dd36b6afa9885fbdeadf"
server_key_version: "encryptionkey@0"
I20260809 12:57:04.367291 30506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:04.385526 30506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:04.389824 30506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:04.390116 30506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:04.391181 30506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:04.392671 30506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:04.392791 30506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:04.392925 30506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:04.393029 30506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:04.428946 30506 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:40791
I20260809 12:57:04.429055 30652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:40791 every 8 connection(s)
I20260809 12:57:04.430497 30506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:57:04.436129 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 30506
I20260809 12:57:04.436486 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 29776
Aug 09 12:57:04 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280224, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.124@KRBTEST.COM
Aug 09 12:57:04 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280224, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM
Aug 09 12:57:04 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.235.67@KRBTEST.COM
I20260809 12:57:04.479598 30657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:04.433653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:59823 (local address 127.21.235.124:40403)
0809 12:57:04.446823 (+ 13170us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:04.446828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:04.446846 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:57:04.451290 (+ 4444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:04.451297 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:04.451428 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:04.451556 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:04.460454 (+ 8898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:04.461633 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:04.464047 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:04.464718 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:04.464804 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:04.470992 (+ 6188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:04.470998 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:04.471003 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:04.471045 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:04.475193 (+ 4148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:04.476158 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:04.476165 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:04.476169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:04.476316 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:04.476895 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:04.476903 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:04.476906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:04.477280 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:04.477408 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:04.478528 (+ 1120us) server_negotiation.cc:299] Negotiation successful
0809 12:57:04.478896 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13059,"thread_start_us":12957,"threads_started":1}
I20260809 12:57:04.480250 30667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:04.451372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:55397 (local address 127.21.235.126:33521)
0809 12:57:04.457339 (+ 5967us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:04.457345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:04.457362 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:57:04.457480 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:04.457489 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:04.457619 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:04.457756 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:04.459724 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:04.461295 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:04.464646 (+ 3351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:04.465623 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:04.465738 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:04.471036 (+ 5298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:04.471043 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:04.471047 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:04.471107 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:04.475726 (+ 4619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:04.476512 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:04.476517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:04.476520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:04.476668 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:04.477133 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:04.477138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:04.477140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:04.477510 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:04.477641 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:04.479088 (+ 1447us) server_negotiation.cc:299] Negotiation successful
0809 12:57:04.479306 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5848,"thread_start_us":4941,"threads_started":1}
I20260809 12:57:04.483179 30665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:04.437676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37314 (local address 127.21.235.67:39729)
0809 12:57:04.454568 (+ 16892us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:04.454580 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:04.454617 (+ 37us) server_negotiation.cc:407] Connection header received
0809 12:57:04.454711 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:04.454723 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:04.455033 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:04.455241 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:04.458019 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:04.459683 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:04.461859 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:04.463257 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:04.463406 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:04.472527 (+ 9121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:04.472543 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:04.472561 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:04.472603 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:04.477840 (+ 5237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:04.479410 (+ 1570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:04.479420 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:04.479424 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:04.479596 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:04.480184 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:04.480189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:04.480193 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:04.480615 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:04.482086 (+ 1471us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:04.482286 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 12:57:04.482617 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16628,"thread_start_us":17797,"threads_started":1}
I20260809 12:57:04.483716 30660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:04.434423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:40403 (local address 127.21.235.66:59823)
0809 12:57:04.443618 (+ 9195us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:04.443696 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:04.450734 (+ 7038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:04.452354 (+ 1620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:04.452374 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:04.454280 (+ 1906us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:04.459737 (+ 5457us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x61200002c628. stack: 00007f3facc8966e 00007f3facc89430 00007f3fabd105ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f3facc7f0fd 00007f3facc7ed1e 00007f3fbfb883cd 00007f3fbfb8836c 00007f3faddb8757 00007f3fb368d03a 00007f3fb3714dca 00007f3fb3712f68 00007f3fb374682f 00007f3fb374658d
0809 12:57:04.460248 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:04.460263 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:04.461944 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:04.461954 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:04.463860 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:04.463874 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:04.465467 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:04.465473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:04.465646 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:04.466277 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:04.466335 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:04.470718 (+ 4383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:04.475370 (+ 4652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:04.475380 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:04.475400 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:04.476021 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:04.476488 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:04.476492 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:04.476495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:04.476764 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:04.477507 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:04.477516 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:04.478005 (+ 489us) client_negotiation.cc:769] Sending connection context
0809 12:57:04.480802 (+ 2797us) client_negotiation.cc:240] Negotiation successful
0809 12:57:04.481587 (+ 785us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1151,"mutex_wait_us":255,"spinlock_wait_cycles":8212480,"thread_start_us":942,"threads_started":1}
I20260809 12:57:04.484920 29915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:04.485268 30666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:04.451227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.66:55397)
0809 12:57:04.456962 (+ 5735us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:04.456982 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:04.457110 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:04.457832 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:04.457840 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:04.458583 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:04.459033 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:04.459048 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:04.461525 (+ 2477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:04.461532 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:04.464475 (+ 2943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:04.464490 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:04.466003 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:04.466008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:04.466227 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:04.466805 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:04.466836 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:04.470718 (+ 3882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:04.475879 (+ 5161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:04.475883 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:04.475887 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:04.476362 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:04.476805 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:04.476813 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:04.476817 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:04.477001 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:04.477695 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:04.477700 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:04.478005 (+ 305us) client_negotiation.cc:769] Sending connection context
0809 12:57:04.480802 (+ 2797us) client_negotiation.cc:240] Negotiation successful
0809 12:57:04.481463 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5558,"mutex_wait_us":70,"spinlock_wait_cycles":4864,"thread_start_us":5401,"threads_started":1}
Aug 09 12:57:04 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280224, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.125@KRBTEST.COM
I20260809 12:57:04.486611 30654 heartbeater.cc:347] Connected to a master server at 127.21.235.126:33521
I20260809 12:57:04.486720 30653 heartbeater.cc:347] Connected to a master server at 127.21.235.124:40403
I20260809 12:57:04.487010 30654 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:04.488412 30654 heartbeater.cc:511] Master 127.21.235.126:33521 requested a full tablet report, sending...
I20260809 12:57:04.487013 30653 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:04.489867 30653 heartbeater.cc:511] Master 127.21.235.124:40403 requested a full tablet report, sending...
I20260809 12:57:04.491336 30298 ts_manager.cc:194] Registered new tserver with Master: 86e19c19da6246d89330b6a6f23b0c50 (127.21.235.66:40791)
I20260809 12:57:04.491571 29402 ts_manager.cc:194] Re-registered known tserver with Master: 86e19c19da6246d89330b6a6f23b0c50 (127.21.235.66:40791)
I20260809 12:57:04.493100 29402 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:55397
I20260809 12:57:04.496659 30661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:04.434398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:42291 (local address 127.21.235.66:56881)
0809 12:57:04.450954 (+ 16556us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:04.450973 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:04.455779 (+ 4806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:04.456215 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:04.456220 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:04.456926 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:04.458019 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:04.458047 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:04.459699 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:04.459705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:04.462771 (+ 3066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:04.462788 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:04.463964 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:04.463974 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:04.464344 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:04.482578 (+ 18234us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:04.482603 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:04.487624 (+ 5021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:04.493516 (+ 5892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:04.493524 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:04.493528 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:04.494489 (+ 961us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:04.494940 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:04.494946 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:04.494950 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:04.495104 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:04.495789 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:04.495794 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:04.495951 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 12:57:04.496097 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 12:57:04.496343 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16350,"mutex_wait_us":473,"thread_start_us":15811,"threads_started":1}
I20260809 12:57:04.497136 30659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:04.433741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:56881 (local address 127.21.235.125:42291)
0809 12:57:04.446346 (+ 12605us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:04.446356 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:04.451069 (+ 4713us) server_negotiation.cc:407] Connection header received
0809 12:57:04.455951 (+ 4882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:04.455959 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:04.456070 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:04.456197 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:04.458210 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:04.459484 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:04.462925 (+ 3441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:04.463722 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:04.463866 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:04.488010 (+ 24144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:04.488018 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:04.488023 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:04.488083 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:04.492985 (+ 4902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:04.494635 (+ 1650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:04.494642 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:04.494646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:04.494812 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:04.495245 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:04.495250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:04.495255 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:04.495651 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:04.495821 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:04.496490 (+ 669us) server_negotiation.cc:299] Negotiation successful
0809 12:57:04.496761 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12435,"thread_start_us":12653,"threads_started":1}
I20260809 12:57:04.497970 30655 heartbeater.cc:347] Connected to a master server at 127.21.235.125:42291
I20260809 12:57:04.498132 30655 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:04.498670 30655 heartbeater.cc:511] Master 127.21.235.125:42291 requested a full tablet report, sending...
I20260809 12:57:04.499763 30085 ts_manager.cc:194] Registered new tserver with Master: 86e19c19da6246d89330b6a6f23b0c50 (127.21.235.66:40791)
I20260809 12:57:04.582427 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 29776
W20260809 12:57:04.600871 29386 connection.cc:535] server connection from 127.21.235.67:57007 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 12:57:04.601754 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:39729
--local_ip_for_outbound_sockets=127.21.235.67
--tserver_master_addrs=127.21.235.126:33521
--webserver_port=43807
--webserver_interface=127.21.235.67
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.67.keytab
--principal=oryx/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:44997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.235.126:33521,127.21.235.125:42291
--tserver_master_addrs=127.21.235.126:33521,127.21.235.125:42291,127.21.235.124:40403 with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:57:04.830722 30495 heartbeater.cc:503] Master 127.21.235.126:33521 was elected leader, sending a full tablet report...
W20260809 12:57:04.941429 30669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:04.941752 30669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:04.941927 30669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:04.949311 30669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:04.949484 30669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:04.949662 30669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:57:04.959539 30669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:44997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.67.keytab
--principal=oryx/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:39729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=43807
--tserver_master_addrs=127.21.235.126:33521,127.21.235.125:42291,127.21.235.124:40403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:04.960872 30669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:04.962218 30669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:04.976603 30675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:04.976519 30679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:04.976557 30676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:04.977198 30669 server_base.cc:1034] running on GCE node
I20260809 12:57:04.977977 30669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:04.979048 30669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:04.980346 30669 hybrid_clock.cc:648] HybridClock initialized: now 1786280224980283 us; error 34 us; skew 500 ppm
I20260809 12:57:04.980738 30669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:04 dist-test-slave-gp6n krb5kdc[29338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280224, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:04.984591 30669 init.cc:377] Logged in from keytab as oryx/127.21.235.67@KRBTEST.COM (short username oryx)
I20260809 12:57:04.986470 30669 webserver.cc:460] Webserver started at http://127.21.235.67:43807/ using document root <none> and password file <none>
I20260809 12:57:04.987197 30669 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:04.987361 30669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:04.993031 30669 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 12:57:04.995957 30688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:04.997352 30669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:57:04.997565 30669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "3865e20ae461466f93bdc56fa5d870b5"
format_stamp: "Formatted at 2026-08-09 12:57:01 on dist-test-slave-gp6n"
server_key: "14866281d012a68a0318ebdd072f01c4"
server_key_iv: "a9ee10fa50fcb9c4edb2c92206c8e361"
server_key_version: "encryptionkey@0"
I20260809 12:57:04.998646 30669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:05.013375 30669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:05.018877 30669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:05.019138 30669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:05.020099 30669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:05.021517 30669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:05.021601 30669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:05.021674 30669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:05.021733 30669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:05.056485 30669 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:39729
I20260809 12:57:05.056595 30813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:39729 every 8 connection(s)
I20260809 12:57:05.057963 30669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:57:05.067016 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 30669
Aug 09 12:57:05 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280224, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM
Aug 09 12:57:05 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280224, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.125@KRBTEST.COM
Aug 09 12:57:05 dist-test-slave-gp6n krb5kdc[29338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280224, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.124@KRBTEST.COM
I20260809 12:57:05.116041 30827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.075123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:32845 (local address 127.21.235.126:33521)
0809 12:57:05.079764 (+ 4641us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:05.079774 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:05.085777 (+ 6003us) server_negotiation.cc:407] Connection header received
0809 12:57:05.085971 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:05.085978 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:05.086067 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:05.086176 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:05.089826 (+ 3650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.091232 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.093971 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.094889 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.095038 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.102379 (+ 7341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:05.102390 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:05.102413 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:05.102467 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:05.109576 (+ 7109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.111497 (+ 1921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.111503 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.111507 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.111756 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.112506 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.112514 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.112520 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.113000 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:05.113161 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:05.115138 (+ 1977us) server_negotiation.cc:299] Negotiation successful
0809 12:57:05.115448 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4475,"thread_start_us":4316,"threads_started":1}
I20260809 12:57:05.124872 30819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.061072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:44091 (local address 127.21.235.125:42291)
0809 12:57:05.068178 (+ 7106us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:05.068189 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:05.085396 (+ 17207us) server_negotiation.cc:407] Connection header received
0809 12:57:05.086205 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:05.086211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:05.086293 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:05.086389 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:05.091074 (+ 4685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.092880 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.098303 (+ 5423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.099410 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.099544 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.106369 (+ 6825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:05.106380 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:05.106388 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:05.106492 (+ 104us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:05.112050 (+ 5558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.119815 (+ 7765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.119823 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.119829 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.120041 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.120777 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.120784 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.120788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.121205 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:05.121363 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:05.123884 (+ 2521us) server_negotiation.cc:299] Negotiation successful
0809 12:57:05.124160 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6923,"thread_start_us":7992,"threads_started":1}
I20260809 12:57:05.126835 30820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.061189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:44921 (local address 127.21.235.124:40403)
0809 12:57:05.080855 (+ 19666us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:05.080863 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:05.080890 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:57:05.081793 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:05.081802 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:05.081938 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:05.082102 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:05.086134 (+ 4032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.087474 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.095263 (+ 7789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.096571 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.096718 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.114451 (+ 17733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:05.114461 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:05.114467 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:05.114523 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:05.118975 (+ 4452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.122012 (+ 3037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.122018 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.122021 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.122185 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.122719 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.122726 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.122735 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.123169 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:05.123306 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:05.126112 (+ 2806us) server_negotiation.cc:299] Negotiation successful
0809 12:57:05.126356 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19518,"thread_start_us":19707,"threads_started":1}
I20260809 12:57:05.127609 30822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.075012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.67:32845)
0809 12:57:05.085645 (+ 10633us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:05.085677 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:05.085822 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:05.088067 (+ 2245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:05.088074 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:05.089106 (+ 1032us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:05.089590 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:05.089606 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.091448 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.091456 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:05.093742 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:05.093762 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.095426 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.095436 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:05.095841 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.096828 (+ 987us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:05.096867 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:05.101983 (+ 5116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:05.109890 (+ 7907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:05.109904 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:05.109929 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:05.111230 (+ 1301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:05.111923 (+ 693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:05.111931 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:05.111937 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:05.112302 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:05.113262 (+ 960us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:05.113274 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:05.113904 (+ 630us) client_negotiation.cc:769] Sending connection context
0809 12:57:05.114357 (+ 453us) client_negotiation.cc:240] Negotiation successful
0809 12:57:05.124545 (+ 10188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10508,"spinlock_wait_cycles":4480,"thread_start_us":9927,"threads_started":1}
I20260809 12:57:05.128628 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.061689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:42291 (local address 127.21.235.67:44091)
0809 12:57:05.085262 (+ 23573us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:05.085288 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:05.086032 (+ 744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:05.086622 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:05.086626 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:05.087528 (+ 902us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:05.090865 (+ 3337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:05.090879 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.093146 (+ 2267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.093154 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:05.095068 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:05.095083 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.099962 (+ 4879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.099971 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:05.100225 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.102155 (+ 1930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:05.102184 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:05.106121 (+ 3937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:05.113235 (+ 7114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:05.113239 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:05.113243 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:05.114164 (+ 921us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:05.120218 (+ 6054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:05.120224 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:05.120231 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:05.120434 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:05.121469 (+ 1035us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:05.121475 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:05.121678 (+ 203us) client_negotiation.cc:769] Sending connection context
0809 12:57:05.125175 (+ 3497us) client_negotiation.cc:240] Negotiation successful
0809 12:57:05.125651 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23409,"mutex_wait_us":2892,"spinlock_wait_cycles":56576,"thread_start_us":11803,"threads_started":1}
I20260809 12:57:05.129112 30823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.061689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:40403 (local address 127.21.235.67:44921)
0809 12:57:05.074221 (+ 12532us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:05.074296 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:05.081239 (+ 6943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:05.082705 (+ 1466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:05.082722 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:05.084044 (+ 1322us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:05.085881 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:05.085916 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.087744 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.087753 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:05.090697 (+ 2944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:05.090717 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.097202 (+ 6485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.097212 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:05.097791 (+ 579us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.100638 (+ 2847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:05.100669 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:05.113011 (+ 12342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:05.119167 (+ 6156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:05.119171 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:05.119175 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:05.120614 (+ 1439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:05.122342 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:05.122347 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:05.122351 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:05.122568 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:05.123396 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:05.123401 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:05.123576 (+ 175us) client_negotiation.cc:769] Sending connection context
0809 12:57:05.127181 (+ 3605us) client_negotiation.cc:240] Negotiation successful
0809 12:57:05.127921 (+ 740us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12108,"mutex_wait_us":5,"thread_start_us":23516,"threads_started":1}
I20260809 12:57:05.130931 30815 heartbeater.cc:347] Connected to a master server at 127.21.235.124:40403
I20260809 12:57:05.131206 30817 heartbeater.cc:347] Connected to a master server at 127.21.235.125:42291
I20260809 12:57:05.131310 30815 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:05.131405 30817 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:05.132429 30816 heartbeater.cc:347] Connected to a master server at 127.21.235.126:33521
I20260809 12:57:05.132596 30816 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:05.133042 30815 heartbeater.cc:511] Master 127.21.235.124:40403 requested a full tablet report, sending...
I20260809 12:57:05.133620 30816 heartbeater.cc:511] Master 127.21.235.126:33521 requested a full tablet report, sending...
I20260809 12:57:05.134557 30817 heartbeater.cc:511] Master 127.21.235.125:42291 requested a full tablet report, sending...
I20260809 12:57:05.136096 30085 ts_manager.cc:194] Registered new tserver with Master: 3865e20ae461466f93bdc56fa5d870b5 (127.21.235.67:39729)
I20260809 12:57:05.136108 29402 ts_manager.cc:194] Re-registered known tserver with Master: 3865e20ae461466f93bdc56fa5d870b5 (127.21.235.67:39729)
I20260809 12:57:05.137399 29402 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:32845
I20260809 12:57:05.138150 30298 ts_manager.cc:194] Registered new tserver with Master: 3865e20ae461466f93bdc56fa5d870b5 (127.21.235.67:39729)
I20260809 12:57:05.143265 30827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.087163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40750 (local address 127.21.235.126:33521)
0809 12:57:05.116341 (+ 29178us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:05.116349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:05.116373 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:57:05.116435 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:05.116443 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:05.116530 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:05.120417 (+ 3887us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:05.120607 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.122241 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.126211 (+ 3970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.129643 (+ 3432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.130917 (+ 1274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.134212 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:05.134219 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:05.134223 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:05.134263 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:05.138994 (+ 4731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.140058 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.140064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.140068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.140228 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.140870 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.140878 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.140883 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.141370 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:05.141853 (+ 483us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:05.142321 (+ 468us) server_negotiation.cc:299] Negotiation successful
0809 12:57:05.142827 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29045,"thread_start_us":38276,"threads_started":1}
I20260809 12:57:05.145515 30831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.086590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44690 (local address 127.21.235.124:40403)
0809 12:57:05.115434 (+ 28844us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:05.115441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:05.115474 (+ 33us) server_negotiation.cc:407] Connection header received
0809 12:57:05.115537 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:05.115545 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:05.115625 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:05.122211 (+ 6586us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:05.122518 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.123817 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.128192 (+ 4375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.129061 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.129843 (+ 782us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.133553 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:05.133562 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:05.133568 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:05.133609 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:05.141419 (+ 7810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.142358 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.142363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.142366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.142580 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.143032 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.143037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.143041 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.143481 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:05.144733 (+ 1252us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:05.144867 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 12:57:05.145112 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28699,"thread_start_us":28517,"threads_started":1}
I20260809 12:57:05.147024 30836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.095410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53842 (local address 127.21.235.125:42291)
0809 12:57:05.107890 (+ 12480us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:05.107897 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:05.111339 (+ 3442us) server_negotiation.cc:407] Connection header received
0809 12:57:05.111818 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:05.111823 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:05.111913 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:05.112008 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:05.116773 (+ 4765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.118275 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.125841 (+ 7566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.126611 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.126737 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.138200 (+ 11463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:05.138222 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:05.138228 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:05.138436 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:05.143038 (+ 4602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.144110 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.144116 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.144120 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.144349 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.144875 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.144882 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.144886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.145191 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:05.145497 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:05.146280 (+ 783us) server_negotiation.cc:299] Negotiation successful
0809 12:57:05.146559 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12285,"thread_start_us":12485,"threads_started":1}
I20260809 12:57:05.148497 29402 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:57:05.157228 29402 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:57:05.197897 30850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.180721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37254 (local address 127.21.235.66:40791)
0809 12:57:05.187202 (+ 6481us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:05.187215 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:05.187256 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:57:05.187404 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:05.187417 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:05.187772 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:05.190899 (+ 3127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:05.191108 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.192553 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.195655 (+ 3102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.196900 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.197061 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.197287 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:05.197465 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 12:57:05.197625 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6260,"thread_start_us":6134,"threads_started":1}
I20260809 12:57:05.197991 30851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.179979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:40791 (local address 127.0.0.1:37254)
0809 12:57:05.182543 (+ 2564us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:05.182567 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:05.182714 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:05.189740 (+ 7026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:05.189746 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:05.189792 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:05.190636 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:05.190655 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.192732 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.192741 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:05.195440 (+ 2699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:05.195455 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.197097 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.197103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:05.197274 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.197311 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 12:57:05.197396 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 12:57:05.197531 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2359,"mutex_wait_us":30,"thread_start_us":12916,"threads_started":1}
I20260809 12:57:05.200348 30848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.179926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53780 (local address 127.21.235.65:39115)
0809 12:57:05.188584 (+ 8658us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:05.188596 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:05.188633 (+ 37us) server_negotiation.cc:407] Connection header received
0809 12:57:05.188758 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:05.188769 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:05.189116 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:05.189371 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:05.190267 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.191983 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.196905 (+ 4922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.198920 (+ 2015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.199068 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.199268 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:05.199849 (+ 581us) server_negotiation.cc:299] Negotiation successful
0809 12:57:05.200035 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8412,"thread_start_us":8255,"threads_started":1}
I20260809 12:57:05.200803 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.179590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:39115 (local address 127.0.0.1:53780)
0809 12:57:05.181099 (+ 1509us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:05.181121 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:05.181252 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:05.189358 (+ 8106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:05.189363 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:05.189390 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:05.190025 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:05.190042 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.192147 (+ 2105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.192153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:05.196685 (+ 4532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:05.196701 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.199460 (+ 2759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.199467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:05.199642 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.199675 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 12:57:05.199768 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 12:57:05.199903 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1369,"mutex_wait_us":46,"thread_start_us":1211,"threads_started":1}
I20260809 12:57:05.201858 30583 tablet_service.cc:1467] Processing CreateTablet for tablet ec729eb03499422c886415989aa34a63 (DEFAULT_TABLE table=test-table [id=ac8ddff9a564438881f295ab12dd4125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:05.203430 30583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec729eb03499422c886415989aa34a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:05.203473 30852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.181925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37316 (local address 127.21.235.67:39729)
0809 12:57:05.193573 (+ 11648us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:05.193583 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:05.193648 (+ 65us) server_negotiation.cc:407] Connection header received
0809 12:57:05.193758 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:05.193766 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:05.194017 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:05.194224 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:05.195748 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.197784 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.200650 (+ 2866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.202237 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.202391 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.202622 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:05.202977 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 12:57:05.203150 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11468,"thread_start_us":11254,"threads_started":1}
I20260809 12:57:05.203518 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.181775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:39729 (local address 127.0.0.1:37316)
0809 12:57:05.193427 (+ 11652us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:05.193447 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:05.193568 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:05.194215 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:05.194218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:05.194237 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:05.195488 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:05.195504 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.197985 (+ 2481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.197993 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:05.200477 (+ 2484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:05.200496 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.202504 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.202553 (+ 49us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:05.202737 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.202761 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 12:57:05.202861 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 12:57:05.202990 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11511,"mutex_wait_us":42,"thread_start_us":370,"threads_started":1}
I20260809 12:57:05.204710 30422 tablet_service.cc:1467] Processing CreateTablet for tablet ec729eb03499422c886415989aa34a63 (DEFAULT_TABLE table=test-table [id=ac8ddff9a564438881f295ab12dd4125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:05.206794 30422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec729eb03499422c886415989aa34a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:05.207070 30742 tablet_service.cc:1467] Processing CreateTablet for tablet ec729eb03499422c886415989aa34a63 (DEFAULT_TABLE table=test-table [id=ac8ddff9a564438881f295ab12dd4125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:05.208168 30742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec729eb03499422c886415989aa34a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:05.223006 30853 tablet_bootstrap.cc:492] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5: Bootstrap starting.
I20260809 12:57:05.225515 30855 tablet_bootstrap.cc:492] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9: Bootstrap starting.
I20260809 12:57:05.227429 30854 tablet_bootstrap.cc:492] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50: Bootstrap starting.
I20260809 12:57:05.228331 30853 tablet_bootstrap.cc:654] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:05.229990 30855 tablet_bootstrap.cc:654] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:05.230036 30853 log.cc:824] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5: Log is configured to *not* fsync() on all Append() calls
I20260809 12:57:05.231630 30854 tablet_bootstrap.cc:654] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:05.231776 30855 log.cc:824] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9: Log is configured to *not* fsync() on all Append() calls
I20260809 12:57:05.233069 30854 log.cc:824] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50: Log is configured to *not* fsync() on all Append() calls
I20260809 12:57:05.235414 30853 tablet_bootstrap.cc:492] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5: No bootstrap required, opened a new log
I20260809 12:57:05.235747 30853 ts_tablet_manager.cc:1397] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260809 12:57:05.235870 30855 tablet_bootstrap.cc:492] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9: No bootstrap required, opened a new log
I20260809 12:57:05.236135 30855 ts_tablet_manager.cc:1397] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260809 12:57:05.236568 30854 tablet_bootstrap.cc:492] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50: No bootstrap required, opened a new log
I20260809 12:57:05.236842 30854 ts_tablet_manager.cc:1397] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260809 12:57:05.244124 30855 raft_consensus.cc:348] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } } peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } }
I20260809 12:57:05.244529 30855 raft_consensus.cc:374] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:05.244640 30855 raft_consensus.cc:729] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2e31488363744cf90ae9aac7c1883d9, State: Initialized, Role: FOLLOWER
I20260809 12:57:05.245327 30855 consensus_queue.cc:260] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } } peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } }
I20260809 12:57:05.247630 30853 raft_consensus.cc:348] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } } peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } }
I20260809 12:57:05.248065 30853 raft_consensus.cc:374] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:05.248158 30853 raft_consensus.cc:729] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3865e20ae461466f93bdc56fa5d870b5, State: Initialized, Role: FOLLOWER
I20260809 12:57:05.248569 30855 ts_tablet_manager.cc:1428] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260809 12:57:05.248642 30854 raft_consensus.cc:348] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } } peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } }
I20260809 12:57:05.249220 30854 raft_consensus.cc:374] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:05.249341 30854 raft_consensus.cc:729] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86e19c19da6246d89330b6a6f23b0c50, State: Initialized, Role: FOLLOWER
I20260809 12:57:05.250061 30854 consensus_queue.cc:260] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } } peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } }
I20260809 12:57:05.248962 30853 consensus_queue.cc:260] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } } peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } }
I20260809 12:57:05.255098 30816 heartbeater.cc:503] Master 127.21.235.126:33521 was elected leader, sending a full tablet report...
I20260809 12:57:05.256924 30654 heartbeater.cc:503] Master 127.21.235.126:33521 was elected leader, sending a full tablet report...
I20260809 12:57:05.257730 30854 ts_tablet_manager.cc:1428] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50: Time spent starting tablet: real 0.021s user 0.003s sys 0.019s
I20260809 12:57:05.259971 30853 ts_tablet_manager.cc:1428] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5: Time spent starting tablet: real 0.024s user 0.011s sys 0.007s
W20260809 12:57:05.282454 30497 tablet.cc:2369] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:57:05.310746 30818 tablet.cc:2369] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:57:05.402842 30860 raft_consensus.cc:482] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:57:05.403316 30860 raft_consensus.cc:504] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } } peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } }
I20260809 12:57:05.405123 30860 leader_election.cc:288] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2e31488363744cf90ae9aac7c1883d9 (127.21.235.65:39115), 3865e20ae461466f93bdc56fa5d870b5 (127.21.235.67:39729)
I20260809 12:57:05.413177 30848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.405598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:36091 (local address 127.21.235.65:39115)
0809 12:57:05.405875 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:05.405884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:05.406566 (+ 682us) server_negotiation.cc:407] Connection header received
0809 12:57:05.406872 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:05.406879 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:05.406971 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:05.407045 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:05.407841 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.408976 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.411332 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.412157 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.412282 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.412369 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:05.412756 (+ 387us) server_negotiation.cc:299] Negotiation successful
0809 12:57:05.412896 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 12:57:05.413195 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.405573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:39115 (local address 127.21.235.66:36091)
0809 12:57:05.406393 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:05.406474 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:05.406652 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:05.407235 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:05.407239 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:05.407259 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:05.407681 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:05.407695 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.409113 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.409119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:05.411157 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:05.411170 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.412381 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.412389 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:05.412582 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.412606 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 12:57:05.412683 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 12:57:05.412826 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":466,"threads_started":1}
I20260809 12:57:05.413194 30852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.406806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:56027 (local address 127.21.235.67:39729)
0809 12:57:05.407022 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:05.407030 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:05.407336 (+ 306us) server_negotiation.cc:407] Connection header received
0809 12:57:05.407469 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:05.407473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:05.407572 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:05.407703 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:05.408435 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.409288 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.411413 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.412256 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.412434 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.412513 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:05.412813 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 12:57:05.412964 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 12:57:05.413304 30864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.406511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:39729 (local address 127.21.235.66:56027)
0809 12:57:05.407091 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:05.407106 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:05.407358 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:05.407730 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:05.407733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:05.407774 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:05.408167 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:05.408182 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.409480 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.409485 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:05.411248 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:05.411268 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.412498 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.412502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:05.412621 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.412644 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 12:57:05.412767 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 12:57:05.412894 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":332,"threads_started":1}
I20260809 12:57:05.414703 30445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec729eb03499422c886415989aa34a63" candidate_uuid: "86e19c19da6246d89330b6a6f23b0c50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e31488363744cf90ae9aac7c1883d9" is_pre_election: true
I20260809 12:57:05.414734 30765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec729eb03499422c886415989aa34a63" candidate_uuid: "86e19c19da6246d89330b6a6f23b0c50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3865e20ae461466f93bdc56fa5d870b5" is_pre_election: true
I20260809 12:57:05.415258 30445 raft_consensus.cc:2454] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86e19c19da6246d89330b6a6f23b0c50 in term 0.
I20260809 12:57:05.415288 30765 raft_consensus.cc:2454] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86e19c19da6246d89330b6a6f23b0c50 in term 0.
I20260809 12:57:05.416002 30533 leader_election.cc:302] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86e19c19da6246d89330b6a6f23b0c50, e2e31488363744cf90ae9aac7c1883d9; no voters:
I20260809 12:57:05.416422 30860 raft_consensus.cc:2781] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:57:05.416587 30860 raft_consensus.cc:482] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:57:05.416697 30860 raft_consensus.cc:3037] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:05.419550 30860 raft_consensus.cc:504] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } } peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } }
I20260809 12:57:05.420336 30860 leader_election.cc:288] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [CANDIDATE]: Term 1 election: Requested vote from peers e2e31488363744cf90ae9aac7c1883d9 (127.21.235.65:39115), 3865e20ae461466f93bdc56fa5d870b5 (127.21.235.67:39729)
I20260809 12:57:05.420876 30445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec729eb03499422c886415989aa34a63" candidate_uuid: "86e19c19da6246d89330b6a6f23b0c50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e31488363744cf90ae9aac7c1883d9"
I20260809 12:57:05.421099 30445 raft_consensus.cc:3037] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:05.421226 30765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec729eb03499422c886415989aa34a63" candidate_uuid: "86e19c19da6246d89330b6a6f23b0c50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3865e20ae461466f93bdc56fa5d870b5"
I20260809 12:57:05.421487 30765 raft_consensus.cc:3037] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:05.423672 30445 raft_consensus.cc:2454] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86e19c19da6246d89330b6a6f23b0c50 in term 1.
I20260809 12:57:05.424154 30533 leader_election.cc:302] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86e19c19da6246d89330b6a6f23b0c50, e2e31488363744cf90ae9aac7c1883d9; no voters:
I20260809 12:57:05.424175 30765 raft_consensus.cc:2454] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86e19c19da6246d89330b6a6f23b0c50 in term 1.
I20260809 12:57:05.424466 30860 raft_consensus.cc:2781] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:57:05.425203 30860 raft_consensus.cc:686] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [term 1 LEADER]: Becoming Leader. State: Replica: 86e19c19da6246d89330b6a6f23b0c50, State: Running, Role: LEADER
I20260809 12:57:05.425750 30860 consensus_queue.cc:237] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } } peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } }
I20260809 12:57:05.433231 29403 catalog_manager.cc:5495] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86e19c19da6246d89330b6a6f23b0c50 (127.21.235.66). New cstate: current_term: 1 leader_uuid: "86e19c19da6246d89330b6a6f23b0c50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } health_report { overall_health: UNKNOWN } } }
W20260809 12:57:05.434780 30656 tablet.cc:2369] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:57:05.491040 30850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.485234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37266 (local address 127.21.235.66:40791)
0809 12:57:05.485431 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:05.485436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:05.485457 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:57:05.485623 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:05.485627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:05.485722 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:05.485908 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:57:05.486261 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.487244 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.488658 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.489154 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.489284 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.489788 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:57:05.490338 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:57:05.490480 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:05.490621 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 12:57:05.490692 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:57:05.499727 30765 raft_consensus.cc:1261] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5 [term 1 FOLLOWER]: Refusing update from remote peer 86e19c19da6246d89330b6a6f23b0c50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:57:05.499819 30445 raft_consensus.cc:1261] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9 [term 1 FOLLOWER]: Refusing update from remote peer 86e19c19da6246d89330b6a6f23b0c50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:57:05.500808 30865 consensus_queue.cc:1035] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:57:05.501313 30860 consensus_queue.cc:1035] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:57:05.517138 30869 mvcc.cc:205] Tried to move back new op lower bound from 7316603803636178944 to 7316603803348631552. Current Snapshot: MvccSnapshot[applied={T|T < 7316603803636178944}]
I20260809 12:57:05.519759 30870 mvcc.cc:205] Tried to move back new op lower bound from 7316603803636178944 to 7316603803348631552. Current Snapshot: MvccSnapshot[applied={T|T < 7316603803636178944}]
I20260809 12:57:05.535910 29401 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:57:05.536249 29401 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:57:05.543830 29401 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0475d10a6054e859f62c2497daa235c: Sending DeleteTablet for 3 replicas of tablet ec729eb03499422c886415989aa34a63
I20260809 12:57:05.545125 30422 tablet_service.cc:1514] Processing DeleteTablet for tablet ec729eb03499422c886415989aa34a63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:05 UTC) from {username='oryx', principal='oryx/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:53780
I20260809 12:57:05.545125 30742 tablet_service.cc:1514] Processing DeleteTablet for tablet ec729eb03499422c886415989aa34a63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:05 UTC) from {username='oryx', principal='oryx/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:37316
I20260809 12:57:05.545948 30583 tablet_service.cc:1514] Processing DeleteTablet for tablet ec729eb03499422c886415989aa34a63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:05 UTC) from {username='oryx', principal='oryx/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:37254
I20260809 12:57:05.557898 30880 tablet_replica.cc:330] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5: stopping tablet replica
I20260809 12:57:05.558378 30880 raft_consensus.cc:2229] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:57:05.558981 30880 raft_consensus.cc:2258] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:05.561344 30880 ts_tablet_manager.cc:1905] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:05.561898 30883 tablet_replica.cc:330] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50: stopping tablet replica
I20260809 12:57:05.562355 30883 raft_consensus.cc:2229] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:57:05.566988 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 29357
I20260809 12:57:05.567824 30882 tablet_replica.cc:330] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9: stopping tablet replica
I20260809 12:57:05.568195 29437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:05.568307 30882 raft_consensus.cc:2229] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:57:05.568476 30883 raft_consensus.cc:2258] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:05.568874 30882 raft_consensus.cc:2258] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:05.570801 30883 ts_tablet_manager.cc:1905] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:05.570991 30882 ts_tablet_manager.cc:1905] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:05.578253 30882 ts_tablet_manager.cc:1918] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:57:05.578449 30882 log.cc:1196] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/ec729eb03499422c886415989aa34a63
I20260809 12:57:05.579054 30882 ts_tablet_manager.cc:1939] T ec729eb03499422c886415989aa34a63 P e2e31488363744cf90ae9aac7c1883d9: Deleting consensus metadata
I20260809 12:57:05.582763 30880 ts_tablet_manager.cc:1918] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:57:05.582983 30880 log.cc:1196] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/ec729eb03499422c886415989aa34a63
I20260809 12:57:05.583665 30880 ts_tablet_manager.cc:1939] T ec729eb03499422c886415989aa34a63 P 3865e20ae461466f93bdc56fa5d870b5: Deleting consensus metadata
I20260809 12:57:05.578276 30883 ts_tablet_manager.cc:1918] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:57:05.586102 30883 log.cc:1196] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/ec729eb03499422c886415989aa34a63
I20260809 12:57:05.586802 30883 ts_tablet_manager.cc:1939] T ec729eb03499422c886415989aa34a63 P 86e19c19da6246d89330b6a6f23b0c50: Deleting consensus metadata
I20260809 12:57:05.591662 30836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.547959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53852 (local address 127.21.235.125:42291)
0809 12:57:05.548193 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:05.548203 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:05.565593 (+ 17390us) server_negotiation.cc:407] Connection header received
0809 12:57:05.566839 (+ 1246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:05.566845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:05.566965 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:05.567058 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:05.572281 (+ 5223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.573719 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.576503 (+ 2784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.577590 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.577721 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.583497 (+ 5776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:05.583507 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:05.583516 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:05.583572 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:05.588586 (+ 5014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.589463 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.589471 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.589478 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.589681 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.590110 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.590115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.590118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.590544 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:05.590733 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:05.591167 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 12:57:05.591349 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 12:57:05.610384 30831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.566326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44694 (local address 127.21.235.124:40403)
0809 12:57:05.566549 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:05.566554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:05.591811 (+ 25257us) server_negotiation.cc:407] Connection header received
0809 12:57:05.592786 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:05.592795 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:05.592912 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:05.593022 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:05.594855 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.595966 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.598108 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.598894 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.599024 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.602719 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:05.602727 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:05.602734 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:05.602778 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:05.607043 (+ 4265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.607950 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.607956 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.607961 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.608147 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.608708 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.608712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.608716 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.609123 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:05.609372 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:05.609790 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 12:57:05.610051 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 12:57:05.717870 29385 catalog_manager.cc:4841] TS 86e19c19da6246d89330b6a6f23b0c50 (127.21.235.66:40791): tablet ec729eb03499422c886415989aa34a63 (table test-table [id=ac8ddff9a564438881f295ab12dd4125]) successfully deleted
I20260809 12:57:05.718737 29384 catalog_manager.cc:4841] TS e2e31488363744cf90ae9aac7c1883d9 (127.21.235.65:39115): tablet ec729eb03499422c886415989aa34a63 (table test-table [id=ac8ddff9a564438881f295ab12dd4125]) successfully deleted
W20260809 12:57:05.718961 29384 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.21.235.65:39115, user_credentials={real_user=slave}} blocked reactor thread for 137958us
I20260809 12:57:05.719442 29384 catalog_manager.cc:4841] TS 3865e20ae461466f93bdc56fa5d870b5 (127.21.235.67:39729): tablet ec729eb03499422c886415989aa34a63 (table test-table [id=ac8ddff9a564438881f295ab12dd4125]) successfully deleted
I20260809 12:57:05.722815 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 29357
I20260809 12:57:05.775866 30831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.753432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44700 (local address 127.21.235.124:40403)
0809 12:57:05.753689 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:05.753698 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:05.755579 (+ 1881us) server_negotiation.cc:407] Connection header received
0809 12:57:05.755933 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:05.755943 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:05.756056 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:05.756153 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:05.759358 (+ 3205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.760835 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.763923 (+ 3088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.764854 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.765004 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.768948 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:05.768957 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:05.768965 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:05.769019 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:05.772600 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.773244 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.773250 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.773254 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.773392 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.773789 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.773794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.773798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.774190 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:05.774573 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:05.775151 (+ 578us) server_negotiation.cc:299] Negotiation successful
0809 12:57:05.775579 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 12:57:05.777604 30836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:05.754646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53862 (local address 127.21.235.125:42291)
0809 12:57:05.754875 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:05.754884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:05.755898 (+ 1014us) server_negotiation.cc:407] Connection header received
0809 12:57:05.756171 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:05.756178 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:05.756272 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:05.756354 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:05.758928 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.760220 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.762617 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:05.763356 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:05.763466 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:05.769078 (+ 5612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:05.769086 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:05.769093 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:05.769140 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:05.773123 (+ 3983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.774041 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.774046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.774049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.774191 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:05.775585 (+ 1394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:05.775591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:05.775596 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:05.775977 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:05.776205 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:05.776793 (+ 588us) server_negotiation.cc:299] Negotiation successful
0809 12:57:05.777203 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 12:57:06.524641 30894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:06.522637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.66:60077)
0809 12:57:06.524255 (+ 1618us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:06.524405 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1514,"thread_start_us":1398,"threads_started":1}
W20260809 12:57:06.525120 30654 heartbeater.cc:644] Failed to heartbeat to 127.21.235.126:33521 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
I20260809 12:57:06.527458 30895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:06.526520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.65:59255)
0809 12:57:06.527037 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:06.527191 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":272,"threads_started":1}
I20260809 12:57:06.528014 30896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:06.527061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.67:52775)
0809 12:57:06.527656 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:06.527796 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":308,"threads_started":1}
W20260809 12:57:06.528046 30495 heartbeater.cc:644] Failed to heartbeat to 127.21.235.126:33521 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
W20260809 12:57:06.528491 30816 heartbeater.cc:644] Failed to heartbeat to 127.21.235.126:33521 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
I20260809 12:57:07.043211 30897 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e0475d10a6054e859f62c2497daa235c)
I20260809 12:57:07.043591 30897 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false } }
I20260809 12:57:07.044124 30898 raft_consensus.cc:482] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e0475d10a6054e859f62c2497daa235c)
I20260809 12:57:07.044360 30898 raft_consensus.cc:504] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false } }
I20260809 12:57:07.045141 30897 leader_election.cc:288] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e0475d10a6054e859f62c2497daa235c (127.21.235.126:33521), ede96f6f1f564f3c962f6e4080908730 (127.21.235.124:40403)
I20260809 12:57:07.045289 30898 leader_election.cc:288] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e0475d10a6054e859f62c2497daa235c (127.21.235.126:33521), 91d3abc7c1254b84bb14a33b01d586d1 (127.21.235.125:42291)
I20260809 12:57:07.046554 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.045367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.0.0.1:41130)
0809 12:57:07.045940 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.046219 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":283,"threads_started":1}
W20260809 12:57:07.046813 30069 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
I20260809 12:57:07.049459 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.046877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.0.0.1:41132)
0809 12:57:07.047650 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.049132 (+ 1482us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":474,"threads_started":1}
W20260809 12:57:07.049782 30281 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
I20260809 12:57:07.051291 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.050822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.0.0.1:41136)
0809 12:57:07.050976 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.051075 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 12:57:07.051594 30069 leader_election.cc:334] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e0475d10a6054e859f62c2497daa235c (127.21.235.126:33521): Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
I20260809 12:57:07.052099 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.051577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.0.0.1:41150)
0809 12:57:07.051838 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.051946 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 12:57:07.052364 30281 leader_election.cc:334] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e0475d10a6054e859f62c2497daa235c (127.21.235.126:33521): Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
I20260809 12:57:07.054754 30902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.045451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:40403 (local address 127.0.0.1:44704)
0809 12:57:07.046324 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.046343 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:07.046551 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:07.047100 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:07.047105 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:07.047137 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:07.047628 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:07.047648 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.049534 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.049541 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:07.052087 (+ 2546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:07.052106 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.053806 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.053812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:07.054035 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:07.054077 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 12:57:07.054198 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 12:57:07.054346 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"thread_start_us":542,"threads_started":1}
I20260809 12:57:07.055013 30903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.045777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44704 (local address 127.21.235.124:40403)
0809 12:57:07.046705 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:07.046711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:07.046732 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:57:07.046810 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:07.046815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:07.046888 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:07.047002 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:07.047829 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.049314 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.052270 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.053583 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.053769 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:07.053825 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:07.054236 (+ 411us) server_negotiation.cc:299] Negotiation successful
0809 12:57:07.054453 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":661,"threads_started":1}
I20260809 12:57:07.055617 30308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "91d3abc7c1254b84bb14a33b01d586d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ede96f6f1f564f3c962f6e4080908730" is_pre_election: true
I20260809 12:57:07.056042 30308 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91d3abc7c1254b84bb14a33b01d586d1 in term 1.
I20260809 12:57:07.056595 30067 leader_election.cc:302] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91d3abc7c1254b84bb14a33b01d586d1, ede96f6f1f564f3c962f6e4080908730; no voters: e0475d10a6054e859f62c2497daa235c
I20260809 12:57:07.057031 30897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 12:57:07.057245 30897 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e0475d10a6054e859f62c2497daa235c)
I20260809 12:57:07.057380 30897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 1 FOLLOWER]: Advancing to term 2
I20260809 12:57:07.058804 30906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.047822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:42291 (local address 127.0.0.1:53866)
0809 12:57:07.049380 (+ 1558us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.049411 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:07.049764 (+ 353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:07.051761 (+ 1997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:07.051766 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:07.051798 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:07.052132 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:07.052146 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.053804 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.053810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:07.056167 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:07.056182 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.058030 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.058036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:07.058224 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:07.058259 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 12:57:07.058353 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 12:57:07.058503 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1306,"thread_start_us":1013,"threads_started":1}
I20260809 12:57:07.058996 30907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.048253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53866 (local address 127.21.235.125:42291)
0809 12:57:07.051148 (+ 2895us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:07.051154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:07.051180 (+ 26us) server_negotiation.cc:407] Connection header received
0809 12:57:07.051232 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:07.051237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:07.051310 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:07.052284 (+ 974us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:07.052520 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.053628 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.056399 (+ 2771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.057819 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.058042 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:07.058121 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:07.058397 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 12:57:07.058575 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2748,"thread_start_us":2149,"threads_started":1}
I20260809 12:57:07.059759 30097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ede96f6f1f564f3c962f6e4080908730" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "91d3abc7c1254b84bb14a33b01d586d1" is_pre_election: true
I20260809 12:57:07.060787 30897 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false } }
I20260809 12:57:07.061398 30097 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ede96f6f1f564f3c962f6e4080908730 in current term 2: Already voted for candidate 91d3abc7c1254b84bb14a33b01d586d1 in this term.
I20260809 12:57:07.061784 30897 leader_election.cc:288] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [CANDIDATE]: Term 2 election: Requested vote from peers e0475d10a6054e859f62c2497daa235c (127.21.235.126:33521), ede96f6f1f564f3c962f6e4080908730 (127.21.235.124:40403)
I20260809 12:57:07.062755 30308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "91d3abc7c1254b84bb14a33b01d586d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ede96f6f1f564f3c962f6e4080908730"
I20260809 12:57:07.063009 30308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 1 FOLLOWER]: Advancing to term 2
I20260809 12:57:07.063406 30902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.063032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.0.0.1:41172)
0809 12:57:07.063174 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.063261 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 12:57:07.063463 30281 leader_election.cc:302] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ede96f6f1f564f3c962f6e4080908730; no voters: 91d3abc7c1254b84bb14a33b01d586d1, e0475d10a6054e859f62c2497daa235c
I20260809 12:57:07.064332 30902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.063949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.0.0.1:41188)
0809 12:57:07.064094 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.064185 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 12:57:07.064518 30069 leader_election.cc:334] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e0475d10a6054e859f62c2497daa235c (127.21.235.126:33521): Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
I20260809 12:57:07.065680 30308 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91d3abc7c1254b84bb14a33b01d586d1 in term 2.
I20260809 12:57:07.065954 30898 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 12:57:07.066109 30067 leader_election.cc:302] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91d3abc7c1254b84bb14a33b01d586d1, ede96f6f1f564f3c962f6e4080908730; no voters: e0475d10a6054e859f62c2497daa235c
I20260809 12:57:07.066394 30897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 12:57:07.066993 30897 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [term 2 LEADER]: Becoming Leader. State: Replica: 91d3abc7c1254b84bb14a33b01d586d1, State: Running, Role: LEADER
I20260809 12:57:07.067378 30897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false } }
I20260809 12:57:07.069154 30910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91d3abc7c1254b84bb14a33b01d586d1. Latest consensus state: current_term: 2 leader_uuid: "91d3abc7c1254b84bb14a33b01d586d1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false } } }
I20260809 12:57:07.069502 30910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:07.070113 30912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:57:07.074231 30912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:57:07.075091 30912 catalog_manager.cc:1259] Loaded cluster ID: 9ea99b7009ef49d794f2c37b68465cb1
I20260809 12:57:07.075196 30912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:57:07.076356 30912 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:57:07.077179 30912 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: Loaded TSK: 0
I20260809 12:57:07.078166 30912 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:57:07.128834 30084 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:57:07.136013 30084 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:57:07.140172 30902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.139730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.0.0.1:41210)
0809 12:57:07.139876 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.140009 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 12:57:07.140296 30308 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [term 2 FOLLOWER]: Refusing update from remote peer 91d3abc7c1254b84bb14a33b01d586d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 12:57:07.141125 30910 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 12:57:07.141296 30902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.140841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.0.0.1:41212)
0809 12:57:07.141045 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.141127 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260809 12:57:07.141448 30069 consensus_peers.cc:487] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 -> Peer e0475d10a6054e859f62c2497daa235c (127.21.235.126:33521): Couldn't send request to peer e0475d10a6054e859f62c2497daa235c. Status: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 12:57:07.143877 30898 sys_catalog.cc:455] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91d3abc7c1254b84bb14a33b01d586d1. Latest consensus state: current_term: 2 leader_uuid: "91d3abc7c1254b84bb14a33b01d586d1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false } } }
I20260809 12:57:07.144188 30898 sys_catalog.cc:458] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:57:07.145593 30897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "91d3abc7c1254b84bb14a33b01d586d1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false } } }
I20260809 12:57:07.145942 30897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1 [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:07.146334 30898 sys_catalog.cc:455] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "91d3abc7c1254b84bb14a33b01d586d1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e0475d10a6054e859f62c2497daa235c" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33521 } } peers { permanent_uuid: "91d3abc7c1254b84bb14a33b01d586d1" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 42291 } attrs { promote: false } } peers { permanent_uuid: "ede96f6f1f564f3c962f6e4080908730" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 40403 } attrs { promote: false } } }
I20260809 12:57:07.146616 30898 sys_catalog.cc:458] T 00000000000000000000000000000000 P ede96f6f1f564f3c962f6e4080908730 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:57:07.161167 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.154642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:39115 (local address 127.0.0.1:53794)
0809 12:57:07.154824 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.154842 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:07.154959 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:07.156102 (+ 1143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:07.156109 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:07.156142 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:07.156672 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:07.156689 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.157808 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.157815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:07.159237 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:07.159249 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.160392 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.160399 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:07.160574 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:07.160604 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 12:57:07.160682 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 12:57:07.160849 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 12:57:07.161197 30921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.155329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53794 (local address 127.21.235.65:39115)
0809 12:57:07.155770 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:07.155775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:07.155790 (+ 15us) server_negotiation.cc:407] Connection header received
0809 12:57:07.155847 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:07.155850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:07.155906 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:07.156797 (+ 891us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:07.156958 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.157678 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.159396 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.160163 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.160323 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:07.160455 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:07.160781 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 12:57:07.160926 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":1024,"threads_started":1}
I20260809 12:57:07.161945 30422 tablet_service.cc:1467] Processing CreateTablet for tablet 9da5296dd9c34d4d8c269885481c053c (DEFAULT_TABLE table=test-table [id=a792de194b2f46c3b2bf594a61880d87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:07.162392 30422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9da5296dd9c34d4d8c269885481c053c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:07.168471 30924 tablet_bootstrap.cc:492] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9: Bootstrap starting.
I20260809 12:57:07.171180 30924 tablet_bootstrap.cc:654] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:07.173637 30924 tablet_bootstrap.cc:492] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9: No bootstrap required, opened a new log
I20260809 12:57:07.173820 30924 ts_tablet_manager.cc:1397] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 12:57:07.174196 30922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.155771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:40791 (local address 127.0.0.1:37280)
0809 12:57:07.156426 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.156450 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:07.156746 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:07.167404 (+ 10658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:07.167410 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:07.167443 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:07.167815 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:07.167833 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.169417 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.169424 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:07.171750 (+ 2326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:07.171764 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.173316 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.173321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:07.173543 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:07.173580 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 12:57:07.173651 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 12:57:07.173808 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":680,"threads_started":1}
I20260809 12:57:07.174744 30924 raft_consensus.cc:348] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } } peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } }
I20260809 12:57:07.175045 30924 raft_consensus.cc:374] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:07.175153 30924 raft_consensus.cc:729] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2e31488363744cf90ae9aac7c1883d9, State: Initialized, Role: FOLLOWER
I20260809 12:57:07.175462 30924 consensus_queue.cc:260] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } } peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } }
I20260809 12:57:07.175729 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.156204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37280 (local address 127.21.235.66:40791)
0809 12:57:07.166699 (+ 10495us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:07.166710 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:07.166739 (+ 29us) server_negotiation.cc:407] Connection header received
0809 12:57:07.166857 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:07.166866 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:07.166996 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:07.167380 (+ 384us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:07.168091 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.169253 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.171895 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.173028 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.175062 (+ 2034us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:07.175178 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:07.175236 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 12:57:07.175397 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10301,"thread_start_us":11362,"threads_started":1}
I20260809 12:57:07.176544 30583 tablet_service.cc:1467] Processing CreateTablet for tablet 9da5296dd9c34d4d8c269885481c053c (DEFAULT_TABLE table=test-table [id=a792de194b2f46c3b2bf594a61880d87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:07.176753 30924 ts_tablet_manager.cc:1428] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 12:57:07.177030 30583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9da5296dd9c34d4d8c269885481c053c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:07.177290 30902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.154154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:39729 (local address 127.0.0.1:37318)
0809 12:57:07.154296 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.154314 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:07.154491 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:07.171364 (+ 16873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:07.171370 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:07.171409 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:07.171656 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:07.171671 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.173259 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.173287 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:07.175116 (+ 1829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:07.175130 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.176599 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.176606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:07.176800 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:07.176836 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 12:57:07.176922 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 12:57:07.177064 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 12:57:07.178124 30920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.154303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37318 (local address 127.21.235.67:39729)
0809 12:57:07.170799 (+ 16496us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:07.170806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:07.170828 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:57:07.170931 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:07.170940 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:07.171043 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:07.171838 (+ 795us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:07.172042 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.173061 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.175304 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.176408 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.177523 (+ 1115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:07.177636 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:07.177687 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 12:57:07.177827 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16351,"thread_start_us":16852,"threads_started":1}
I20260809 12:57:07.179076 30742 tablet_service.cc:1467] Processing CreateTablet for tablet 9da5296dd9c34d4d8c269885481c053c (DEFAULT_TABLE table=test-table [id=a792de194b2f46c3b2bf594a61880d87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:07.179505 30742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9da5296dd9c34d4d8c269885481c053c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:07.189944 30927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.177327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.65:56953)
0809 12:57:07.189567 (+ 12240us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.189717 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12115,"threads_started":1}
I20260809 12:57:07.191406 30927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.190958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.65:40767)
0809 12:57:07.191181 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.191264 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260809 12:57:07.192282 30929 tablet_bootstrap.cc:492] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5: Bootstrap starting.
I20260809 12:57:07.192811 30928 tablet_bootstrap.cc:492] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50: Bootstrap starting.
I20260809 12:57:07.193383 30496 heartbeater.cc:503] Master 127.21.235.125:42291 was elected leader, sending a full tablet report...
I20260809 12:57:07.194384 30929 tablet_bootstrap.cc:654] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:07.195531 30928 tablet_bootstrap.cc:654] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:07.197054 30929 tablet_bootstrap.cc:492] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5: No bootstrap required, opened a new log
I20260809 12:57:07.197217 30929 ts_tablet_manager.cc:1397] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260809 12:57:07.197664 30928 tablet_bootstrap.cc:492] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50: No bootstrap required, opened a new log
I20260809 12:57:07.197826 30928 ts_tablet_manager.cc:1397] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 12:57:07.198019 30929 raft_consensus.cc:348] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } } peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } }
I20260809 12:57:07.198217 30929 raft_consensus.cc:374] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:07.198333 30929 raft_consensus.cc:729] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3865e20ae461466f93bdc56fa5d870b5, State: Initialized, Role: FOLLOWER
I20260809 12:57:07.198634 30929 consensus_queue.cc:260] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } } peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } }
I20260809 12:57:07.198719 30928 raft_consensus.cc:348] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } } peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } }
I20260809 12:57:07.198977 30928 raft_consensus.cc:374] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:07.199079 30928 raft_consensus.cc:729] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86e19c19da6246d89330b6a6f23b0c50, State: Initialized, Role: FOLLOWER
I20260809 12:57:07.199473 30929 ts_tablet_manager.cc:1428] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 12:57:07.199327 30928 consensus_queue.cc:260] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } } peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } }
I20260809 12:57:07.200945 30928 ts_tablet_manager.cc:1428] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 12:57:07.202347 30935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.201305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.66:54365)
0809 12:57:07.201921 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.202079 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":313,"threads_started":1}
I20260809 12:57:07.203697 30935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.203211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.66:32995)
0809 12:57:07.203452 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.203555 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260809 12:57:07.204704 30655 heartbeater.cc:503] Master 127.21.235.125:42291 was elected leader, sending a full tablet report...
I20260809 12:57:07.207226 30934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.199812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.67:39773)
0809 12:57:07.206844 (+ 7032us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.206999 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6922,"thread_start_us":6819,"threads_started":1}
I20260809 12:57:07.208220 30934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.207820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.67:53251)
0809 12:57:07.207965 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.208077 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 12:57:07.208345 30817 heartbeater.cc:503] Master 127.21.235.125:42291 was elected leader, sending a full tablet report...
I20260809 12:57:07.212581 30933 raft_consensus.cc:482] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:57:07.212752 30933 raft_consensus.cc:504] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } } peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } }
I20260809 12:57:07.213362 30933 leader_election.cc:288] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3865e20ae461466f93bdc56fa5d870b5 (127.21.235.67:39729), e2e31488363744cf90ae9aac7c1883d9 (127.21.235.65:39115)
I20260809 12:57:07.213835 30765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da5296dd9c34d4d8c269885481c053c" candidate_uuid: "86e19c19da6246d89330b6a6f23b0c50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3865e20ae461466f93bdc56fa5d870b5" is_pre_election: true
I20260809 12:57:07.214090 30765 raft_consensus.cc:2454] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86e19c19da6246d89330b6a6f23b0c50 in term 0.
I20260809 12:57:07.214054 30445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da5296dd9c34d4d8c269885481c053c" candidate_uuid: "86e19c19da6246d89330b6a6f23b0c50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e31488363744cf90ae9aac7c1883d9" is_pre_election: true
I20260809 12:57:07.214319 30445 raft_consensus.cc:2454] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86e19c19da6246d89330b6a6f23b0c50 in term 0.
I20260809 12:57:07.214685 30533 leader_election.cc:302] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3865e20ae461466f93bdc56fa5d870b5, 86e19c19da6246d89330b6a6f23b0c50; no voters:
I20260809 12:57:07.214980 30933 raft_consensus.cc:2781] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:57:07.215294 30933 raft_consensus.cc:482] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:57:07.215425 30933 raft_consensus.cc:3037] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:07.217653 30933 raft_consensus.cc:504] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } } peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } }
I20260809 12:57:07.218168 30933 leader_election.cc:288] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [CANDIDATE]: Term 1 election: Requested vote from peers 3865e20ae461466f93bdc56fa5d870b5 (127.21.235.67:39729), e2e31488363744cf90ae9aac7c1883d9 (127.21.235.65:39115)
I20260809 12:57:07.218621 30765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da5296dd9c34d4d8c269885481c053c" candidate_uuid: "86e19c19da6246d89330b6a6f23b0c50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3865e20ae461466f93bdc56fa5d870b5"
I20260809 12:57:07.218866 30765 raft_consensus.cc:3037] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:07.218891 30445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da5296dd9c34d4d8c269885481c053c" candidate_uuid: "86e19c19da6246d89330b6a6f23b0c50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e31488363744cf90ae9aac7c1883d9"
I20260809 12:57:07.219094 30445 raft_consensus.cc:3037] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:07.222028 30765 raft_consensus.cc:2454] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86e19c19da6246d89330b6a6f23b0c50 in term 1.
I20260809 12:57:07.222157 30445 raft_consensus.cc:2454] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86e19c19da6246d89330b6a6f23b0c50 in term 1.
I20260809 12:57:07.222530 30533 leader_election.cc:302] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3865e20ae461466f93bdc56fa5d870b5, 86e19c19da6246d89330b6a6f23b0c50; no voters:
I20260809 12:57:07.222879 30933 raft_consensus.cc:2781] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:57:07.223399 30933 raft_consensus.cc:686] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [term 1 LEADER]: Becoming Leader. State: Replica: 86e19c19da6246d89330b6a6f23b0c50, State: Running, Role: LEADER
I20260809 12:57:07.223707 30933 consensus_queue.cc:237] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } } peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } }
I20260809 12:57:07.224460 30935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.223963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.66:56529)
0809 12:57:07.224125 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.224264 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 12:57:07.226541 30084 catalog_manager.cc:5495] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86e19c19da6246d89330b6a6f23b0c50 (127.21.235.66). New cstate: current_term: 1 leader_uuid: "86e19c19da6246d89330b6a6f23b0c50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86e19c19da6246d89330b6a6f23b0c50" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40791 } health_report { overall_health: HEALTHY } } }
I20260809 12:57:07.255106 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.250256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37292 (local address 127.21.235.66:40791)
0809 12:57:07.250490 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:07.250498 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:07.250537 (+ 39us) server_negotiation.cc:407] Connection header received
0809 12:57:07.250775 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:07.250779 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:07.250846 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:07.250952 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:57:07.251325 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.252045 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.253336 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.253769 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.253914 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:07.254315 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:57:07.254484 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:57:07.254586 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:07.254739 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 12:57:07.254823 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 12:57:07.258426 30445 raft_consensus.cc:1261] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9 [term 1 FOLLOWER]: Refusing update from remote peer 86e19c19da6246d89330b6a6f23b0c50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:57:07.258499 30765 raft_consensus.cc:1261] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5 [term 1 FOLLOWER]: Refusing update from remote peer 86e19c19da6246d89330b6a6f23b0c50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:57:07.259130 30936 consensus_queue.cc:1035] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2e31488363744cf90ae9aac7c1883d9" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:57:07.259478 30933 consensus_queue.cc:1035] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3865e20ae461466f93bdc56fa5d870b5" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:57:07.262351 30927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.261865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.65:34577)
0809 12:57:07.262093 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.262175 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260809 12:57:07.263229 30934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.262828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.67:37155)
0809 12:57:07.262996 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.263098 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 12:57:07.273183 30935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.272458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.66:57813)
0809 12:57:07.272924 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.273013 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354}
I20260809 12:57:07.273425 30934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.272954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.67:51607)
0809 12:57:07.273164 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.273254 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260809 12:57:07.276554 30927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.275645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.65:58625)
0809 12:57:07.276207 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.276345 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411}
I20260809 12:57:07.282439 30935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.281976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.21.235.66:58899)
0809 12:57:07.282120 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.282215 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 12:57:07.283412 30084 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:57:07.283752 30084 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:57:07.290125 30084 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91d3abc7c1254b84bb14a33b01d586d1: Sending DeleteTablet for 3 replicas of tablet 9da5296dd9c34d4d8c269885481c053c
I20260809 12:57:07.291221 30583 tablet_service.cc:1514] Processing DeleteTablet for tablet 9da5296dd9c34d4d8c269885481c053c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:07 UTC) from {username='oryx', principal='oryx/127.21.235.125@KRBTEST.COM'} at 127.0.0.1:37280
I20260809 12:57:07.291390 30742 tablet_service.cc:1514] Processing DeleteTablet for tablet 9da5296dd9c34d4d8c269885481c053c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:07 UTC) from {username='oryx', principal='oryx/127.21.235.125@KRBTEST.COM'} at 127.0.0.1:37318
I20260809 12:57:07.291401 30422 tablet_service.cc:1514] Processing DeleteTablet for tablet 9da5296dd9c34d4d8c269885481c053c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:07 UTC) from {username='oryx', principal='oryx/127.21.235.125@KRBTEST.COM'} at 127.0.0.1:53794
I20260809 12:57:07.291425 30582 tablet_service.cc:1514] Processing DeleteTablet for tablet 9da5296dd9c34d4d8c269885481c053c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:07 UTC) from {username='oryx', principal='oryx/127.21.235.125@KRBTEST.COM'} at 127.0.0.1:37280
I20260809 12:57:07.292898 30949 tablet_replica.cc:330] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50: stopping tablet replica
I20260809 12:57:07.293104 30949 raft_consensus.cc:2229] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:57:07.293524 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 30277
I20260809 12:57:07.294113 30950 tablet_replica.cc:330] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5: stopping tablet replica
I20260809 12:57:07.294394 30950 raft_consensus.cc:2229] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:57:07.294732 30950 raft_consensus.cc:2258] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:07.295560 30949 raft_consensus.cc:2258] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:07.295985 30950 ts_tablet_manager.cc:1905] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:07.296805 30949 ts_tablet_manager.cc:1905] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:07.304420 30951 tablet_replica.cc:330] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9: stopping tablet replica
I20260809 12:57:07.304615 30951 raft_consensus.cc:2229] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:57:07.304939 30951 raft_consensus.cc:2258] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:07.306017 30951 ts_tablet_manager.cc:1905] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:07.307412 30949 ts_tablet_manager.cc:1918] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:57:07.307549 30949 log.cc:1196] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/9da5296dd9c34d4d8c269885481c053c
I20260809 12:57:07.308060 30949 ts_tablet_manager.cc:1939] T 9da5296dd9c34d4d8c269885481c053c P 86e19c19da6246d89330b6a6f23b0c50: Deleting consensus metadata
I20260809 12:57:07.309242 30069 catalog_manager.cc:4841] TS 86e19c19da6246d89330b6a6f23b0c50 (127.21.235.66:40791): tablet 9da5296dd9c34d4d8c269885481c053c (table test-table [id=a792de194b2f46c3b2bf594a61880d87]) successfully deleted
W20260809 12:57:07.309777 30069 catalog_manager.cc:4808] TS 86e19c19da6246d89330b6a6f23b0c50 (127.21.235.66:40791): delete failed for tablet 9da5296dd9c34d4d8c269885481c053c because the tablet was not found. No further retry: Not found: Tablet not found: 9da5296dd9c34d4d8c269885481c053c
I20260809 12:57:07.312507 30951 ts_tablet_manager.cc:1918] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:57:07.312628 30951 log.cc:1196] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/9da5296dd9c34d4d8c269885481c053c
I20260809 12:57:07.313056 30951 ts_tablet_manager.cc:1939] T 9da5296dd9c34d4d8c269885481c053c P e2e31488363744cf90ae9aac7c1883d9: Deleting consensus metadata
I20260809 12:57:07.314064 30068 catalog_manager.cc:4841] TS e2e31488363744cf90ae9aac7c1883d9 (127.21.235.65:39115): tablet 9da5296dd9c34d4d8c269885481c053c (table test-table [id=a792de194b2f46c3b2bf594a61880d87]) successfully deleted
I20260809 12:57:07.314851 30950 ts_tablet_manager.cc:1918] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:57:07.315057 30950 log.cc:1196] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/9da5296dd9c34d4d8c269885481c053c
I20260809 12:57:07.315778 30950 ts_tablet_manager.cc:1939] T 9da5296dd9c34d4d8c269885481c053c P 3865e20ae461466f93bdc56fa5d870b5: Deleting consensus metadata
I20260809 12:57:07.316766 30068 catalog_manager.cc:4841] TS 3865e20ae461466f93bdc56fa5d870b5 (127.21.235.67:39729): tablet 9da5296dd9c34d4d8c269885481c053c (table test-table [id=a792de194b2f46c3b2bf594a61880d87]) successfully deleted
I20260809 12:57:07.321863 30921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.294658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53804 (local address 127.21.235.65:39115)
0809 12:57:07.294872 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:07.294878 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:07.302847 (+ 7969us) server_negotiation.cc:407] Connection header received
0809 12:57:07.303055 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:07.303061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:07.303160 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:07.303253 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:07.306476 (+ 3223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.307580 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.311109 (+ 3529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.311800 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.311910 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:07.315630 (+ 3720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:07.315648 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:07.315669 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:07.315708 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:07.319045 (+ 3337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:07.319892 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:07.319900 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:07.319902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:07.320052 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:07.320427 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:07.320430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:07.320432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:07.320782 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:07.321024 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:07.321319 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 12:57:07.321502 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:57:07.322608 30489 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:07.424245 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 30277
I20260809 12:57:07.447599 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 30506
I20260809 12:57:07.460793 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.448337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.21.235.66:40791)
0809 12:57:07.448792 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:07.448797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:07.448811 (+ 14us) server_negotiation.cc:407] Connection header received
0809 12:57:07.448863 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:07.448866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:07.448969 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:07.449048 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:07.450468 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.451170 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.452378 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.452769 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.452857 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:07.455474 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:07.455489 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:07.455505 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:07.455543 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:07.458221 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:07.458918 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:07.458925 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:07.458928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:07.459059 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:07.459414 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:07.459419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:07.459422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:07.459785 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:07.460015 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:07.460267 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 12:57:07.460429 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347}
I20260809 12:57:07.461472 30647 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:07.554849 30902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.554353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.0.0.1:41214)
0809 12:57:07.554546 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.554662 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 12:57:07.556057 30902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.555682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33521 (local address 127.0.0.1:41226)
0809 12:57:07.555848 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:07.555926 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:33521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 12:57:07.589048 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 30506
I20260809 12:57:07.615733 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 30669
I20260809 12:57:07.630566 30920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:07.616478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37334 (local address 127.21.235.67:39729)
0809 12:57:07.616780 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:07.616785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:07.616805 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:57:07.616864 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:07.616868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:07.616930 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:07.617017 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:07.618533 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.619388 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.620676 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:07.621299 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:07.621454 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:07.624287 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:07.624303 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:07.624319 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:07.624396 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:07.627646 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:07.628409 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:07.628416 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:07.628418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:07.628553 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:07.628907 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:07.628911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:07.628913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:07.629300 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:07.629549 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:07.629931 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 12:57:07.630133 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260809 12:57:07.631263 30808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:07.726595 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 30669
I20260809 12:57:07.746461 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 29929
I20260809 12:57:07.747689 30118 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:07.876057 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 29929
I20260809 12:57:07.903774 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 30043
I20260809 12:57:07.904910 30328 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:08.025213 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 30043
2026-08-09T12:57:08Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9999 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:57:08 dist-test-slave-gp6n krb5kdc[30960](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:57:08 dist-test-slave-gp6n krb5kdc[30960](info): set up 2 sockets
Aug 09 12:57:08 dist-test-slave-gp6n krb5kdc[30960](info): commencing operation
krb5kdc: starting...
W20260809 12:57:09.269629 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:57:09 dist-test-slave-gp6n krb5kdc[30960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:57:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:57:09Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.126.keytab.
Entry for principal oryx/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.126.keytab.
I20260809 12:57:09.530594 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:43003
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.126.keytab
--principal=oryx/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:43003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:57:09.858144 30974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:09.858489 30974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:09.858680 30974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:09.865981 30974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:57:09.866078 30974 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 12:57:09.866171 30974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:09.866279 30974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:09.866369 30974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:57:09.866477 30974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:57:09.876224 30974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:43003
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.126.keytab
--principal=oryx/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:43003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:09.877472 30974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:09.878794 30974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:09.890614 30982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:09.890723 30979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:09.890699 30980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:09.892138 30974 server_base.cc:1034] running on GCE node
I20260809 12:57:09.892877 30974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:09.894163 30974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:09.895460 30974 hybrid_clock.cc:648] HybridClock initialized: now 1786280229895333 us; error 70 us; skew 500 ppm
I20260809 12:57:09.895799 30974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:09 dist-test-slave-gp6n krb5kdc[30960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280229, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:09.898875 30974 init.cc:377] Logged in from keytab as oryx/127.21.235.126@KRBTEST.COM (short username oryx)
I20260809 12:57:09.900365 30974 webserver.cc:460] Webserver started at http://127.21.235.126:46333/ using document root <none> and password file <none>
I20260809 12:57:09.901338 30974 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:09.901487 30974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:09.901830 30974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:09.904865 30974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "6ba10c24ebcc427481e7e33b53805c99"
format_stamp: "Formatted at 2026-08-09 12:57:09 on dist-test-slave-gp6n"
server_key: "b16dc036bfc42a10738c67f42296919b"
server_key_iv: "1d8b4244c827ba23cedb1ef6e7722d94"
server_key_version: "encryptionkey@0"
I20260809 12:57:09.905649 30974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "6ba10c24ebcc427481e7e33b53805c99"
format_stamp: "Formatted at 2026-08-09 12:57:09 on dist-test-slave-gp6n"
server_key: "b16dc036bfc42a10738c67f42296919b"
server_key_iv: "1d8b4244c827ba23cedb1ef6e7722d94"
server_key_version: "encryptionkey@0"
I20260809 12:57:09.911011 30974 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 12:57:09.914345 30989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:09.915773 30974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:57:09.915951 30974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "6ba10c24ebcc427481e7e33b53805c99"
format_stamp: "Formatted at 2026-08-09 12:57:09 on dist-test-slave-gp6n"
server_key: "b16dc036bfc42a10738c67f42296919b"
server_key_iv: "1d8b4244c827ba23cedb1ef6e7722d94"
server_key_version: "encryptionkey@0"
I20260809 12:57:09.916173 30974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:09.941959 30974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:09.946612 30974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:09.946895 30974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:09.968839 30974 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:43003
I20260809 12:57:09.968859 31058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:43003 every 8 connection(s)
I20260809 12:57:09.970245 30974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:57:09.973599 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 30974
I20260809 12:57:09.973819 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:57:09.974546 31059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:09.974711 22445 external_mini_cluster.cc:1428] Setting key 9b47ea1c95ee003a59a64dde08bcbbb1
I20260809 12:57:09.987159 31059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99: Bootstrap starting.
Aug 09 12:57:09 dist-test-slave-gp6n krb5kdc[30960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM
I20260809 12:57:09.990799 31059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:09.992079 31059 log.cc:824] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99: Log is configured to *not* fsync() on all Append() calls
I20260809 12:57:09.995548 31059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99: No bootstrap required, opened a new log
I20260809 12:57:09.999123 31064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:09.976863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48012 (local address 127.21.235.126:43003)
0809 12:57:09.977669 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:09.977681 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:09.977726 (+ 45us) server_negotiation.cc:407] Connection header received
0809 12:57:09.979607 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:09.979631 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:09.980033 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:09.980546 (+ 513us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:09.982128 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:09.983539 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:09.984914 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:09.985671 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:09.985876 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:09.990497 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:09.990514 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:09.990529 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:09.990566 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:09.994074 (+ 3508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:09.995057 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:09.995063 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:09.995070 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:09.995265 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:09.995730 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:09.995736 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:09.995739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:09.996485 (+ 746us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:09.996774 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:09.997192 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 12:57:09.997663 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":354,"threads_started":1}
I20260809 12:57:10.005115 31059 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ba10c24ebcc427481e7e33b53805c99" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43003 } }
I20260809 12:57:10.005457 31059 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:10.005530 31059 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ba10c24ebcc427481e7e33b53805c99, State: Initialized, Role: FOLLOWER
I20260809 12:57:10.006003 31059 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ba10c24ebcc427481e7e33b53805c99" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43003 } }
I20260809 12:57:10.006199 31059 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:57:10.006294 31059 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:57:10.006520 31059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:10.008659 31059 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ba10c24ebcc427481e7e33b53805c99" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43003 } }
I20260809 12:57:10.009094 31059 leader_election.cc:302] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ba10c24ebcc427481e7e33b53805c99; no voters:
I20260809 12:57:10.009688 31059 leader_election.cc:288] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:57:10.009788 31067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:57:10.010847 31067 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99 [term 1 LEADER]: Becoming Leader. State: Replica: 6ba10c24ebcc427481e7e33b53805c99, State: Running, Role: LEADER
I20260809 12:57:10.011436 31067 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ba10c24ebcc427481e7e33b53805c99" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43003 } }
I20260809 12:57:10.011845 31059 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:57:10.014540 31068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ba10c24ebcc427481e7e33b53805c99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ba10c24ebcc427481e7e33b53805c99" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43003 } } }
I20260809 12:57:10.015002 31068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99 [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:10.015625 31069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ba10c24ebcc427481e7e33b53805c99. Latest consensus state: current_term: 1 leader_uuid: "6ba10c24ebcc427481e7e33b53805c99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ba10c24ebcc427481e7e33b53805c99" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43003 } } }
I20260809 12:57:10.016315 31069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99 [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:10.017295 31076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:57:10.022670 31076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:57:10.032148 31076 catalog_manager.cc:1347] Generated new cluster ID: 655ab2feddbf4eb894822a9533315bdb
I20260809 12:57:10.032289 31076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:57:10.046599 31076 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:57:10.047914 31076 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:57:10.061074 31076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99: Generated new TSK 0
I20260809 12:57:10.061998 31076 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.65.keytab.
Entry for principal oryx/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.65.keytab.
I20260809 12:57:10.138339 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:43003
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.65.keytab
--principal=oryx/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:57:10.501688 31088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:10.501958 31088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:10.502135 31088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:10.509366 31088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:10.509469 31088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:10.509642 31088 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:57:10.509764 31088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:57:10.519888 31088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.65.keytab
--principal=oryx/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.235.126:43003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:10.521201 31088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:10.522533 31088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:10.535607 31097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:10.535566 31095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:10.535653 31094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:10.536702 31088 server_base.cc:1034] running on GCE node
I20260809 12:57:10.537271 31088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:10.538133 31088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:10.539347 31088 hybrid_clock.cc:648] HybridClock initialized: now 1786280230539249 us; error 66 us; skew 500 ppm
I20260809 12:57:10.539641 31088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:10 dist-test-slave-gp6n krb5kdc[30960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280230, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:10.542853 31088 init.cc:377] Logged in from keytab as oryx/127.21.235.65@KRBTEST.COM (short username oryx)
I20260809 12:57:10.544221 31088 webserver.cc:460] Webserver started at http://127.21.235.65:41629/ using document root <none> and password file <none>
I20260809 12:57:10.544869 31088 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:10.544956 31088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:10.545219 31088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:10.548101 31088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "1b16ecc6081944b688b42d8afb5a861f"
format_stamp: "Formatted at 2026-08-09 12:57:10 on dist-test-slave-gp6n"
server_key: "b3210bdce972ebc1b9a6e1fd4a5cfd57"
server_key_iv: "d0296f5e0648148535207b22bd0b8a48"
server_key_version: "encryptionkey@0"
I20260809 12:57:10.548856 31088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "1b16ecc6081944b688b42d8afb5a861f"
format_stamp: "Formatted at 2026-08-09 12:57:10 on dist-test-slave-gp6n"
server_key: "b3210bdce972ebc1b9a6e1fd4a5cfd57"
server_key_iv: "d0296f5e0648148535207b22bd0b8a48"
server_key_version: "encryptionkey@0"
I20260809 12:57:10.554100 31088 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:57:10.557607 31107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:10.559029 31088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 12:57:10.559206 31088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "1b16ecc6081944b688b42d8afb5a861f"
format_stamp: "Formatted at 2026-08-09 12:57:10 on dist-test-slave-gp6n"
server_key: "b3210bdce972ebc1b9a6e1fd4a5cfd57"
server_key_iv: "d0296f5e0648148535207b22bd0b8a48"
server_key_version: "encryptionkey@0"
I20260809 12:57:10.559453 31088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:10.584875 31088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:10.589464 31088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:10.589756 31088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:10.592614 31088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:10.592732 31088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:10.592921 31088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:10.593034 31088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:57:10 dist-test-slave-gp6n krb5kdc[30960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280230, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM
I20260809 12:57:10.616551 31116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:10.593026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:55199 (local address 127.21.235.126:43003)
0809 12:57:10.593575 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:10.593584 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:10.594713 (+ 1129us) server_negotiation.cc:407] Connection header received
0809 12:57:10.596921 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:10.596929 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:10.597030 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:10.597169 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:10.599347 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:10.600381 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:10.603072 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:10.603637 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:10.603783 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:10.608680 (+ 4897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:10.608689 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:10.608695 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:10.608742 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:10.612352 (+ 3610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:10.613507 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:10.613517 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:10.613524 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:10.613745 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:10.614221 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:10.614229 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:10.614235 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:10.614724 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:10.614916 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:10.615938 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0809 12:57:10.616169 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":282,"threads_started":1}
I20260809 12:57:10.617615 31119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:10.593727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:43003 (local address 127.21.235.65:55199)
0809 12:57:10.594567 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:10.594608 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:10.596448 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:10.597399 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:10.597413 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:10.598239 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:10.599177 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:10.599197 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:10.600566 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:10.600573 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:10.602909 (+ 2336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:10.602932 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:10.603880 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:10.603888 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:10.604148 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:10.604977 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:10.605019 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:10.608338 (+ 3319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:10.612502 (+ 4164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:10.612510 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:10.612530 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:10.613348 (+ 818us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:10.613881 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:10.613884 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:10.613886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:10.614097 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:10.614879 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:10.614892 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:10.615447 (+ 555us) client_negotiation.cc:769] Sending connection context
0809 12:57:10.615820 (+ 373us) client_negotiation.cc:240] Negotiation successful
0809 12:57:10.616397 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":336,"threads_started":1}
I20260809 12:57:10.644685 31205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:10.630031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:43003 (local address 127.21.235.65:57949)
0809 12:57:10.630684 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:10.630711 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:10.630825 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:10.631200 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:10.631203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:10.631677 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:10.631970 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:10.631979 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:10.633717 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:10.633720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:10.634928 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:10.634947 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:10.636129 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:10.636142 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:10.636305 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:10.637032 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:10.637049 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:10.637799 (+ 750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:10.641523 (+ 3724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:10.641531 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:10.641535 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:10.642317 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:10.642776 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:10.642781 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:10.642788 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:10.643020 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:10.643825 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:10.643828 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:10.643996 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 12:57:10.644104 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 12:57:10.644354 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":338,"threads_started":1}
I20260809 12:57:10.645114 31116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:10.629913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:57949 (local address 127.21.235.126:43003)
0809 12:57:10.630151 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:10.630161 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:10.630810 (+ 649us) server_negotiation.cc:407] Connection header received
0809 12:57:10.630935 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:10.630943 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:10.631050 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:10.631188 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:10.632118 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:10.633573 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:10.635093 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:10.635902 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:10.636065 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:10.638017 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:10.638028 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:10.638036 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:10.638088 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:10.641353 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:10.642483 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:10.642490 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:10.642495 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:10.642659 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:10.643181 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:10.643190 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:10.643197 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:10.643687 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:10.643895 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:10.644285 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 12:57:10.644593 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 12:57:10.653733 31088 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:42239
I20260809 12:57:10.653781 31242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:42239 every 8 connection(s)
I20260809 12:57:10.655030 31088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:57:10.657058 31243 heartbeater.cc:347] Connected to a master server at 127.21.235.126:43003
I20260809 12:57:10.657284 31243 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:10.657922 31243 heartbeater.cc:511] Master 127.21.235.126:43003 requested a full tablet report, sending...
I20260809 12:57:10.658794 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 31088
I20260809 12:57:10.659020 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:57:10.659857 22445 external_mini_cluster.cc:1428] Setting key 990b21f6c358c1eb938ccbd76076d77d
I20260809 12:57:10.660316 31009 ts_manager.cc:194] Registered new tserver with Master: 1b16ecc6081944b688b42d8afb5a861f (127.21.235.65:42239)
I20260809 12:57:10.662300 31009 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:55199
WARNING: no policy specified for oryx/127.21.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.66.keytab.
Entry for principal oryx/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.66.keytab.
I20260809 12:57:10.713346 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:43003
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.66.keytab
--principal=oryx/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:57:11.044180 31248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:11.044488 31248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:11.044591 31248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:11.052157 31248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:11.052273 31248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:11.052366 31248 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:57:11.052451 31248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:57:11.062377 31248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.66.keytab
--principal=oryx/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.235.126:43003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:11.063721 31248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:11.065165 31248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:11.079046 31255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:11.079121 31254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:11.079095 31257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:11.079732 31248 server_base.cc:1034] running on GCE node
I20260809 12:57:11.080363 31248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:11.081275 31248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:11.082537 31248 hybrid_clock.cc:648] HybridClock initialized: now 1786280231082457 us; error 37 us; skew 500 ppm
I20260809 12:57:11.082891 31248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:11 dist-test-slave-gp6n krb5kdc[30960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280231, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:11.086253 31248 init.cc:377] Logged in from keytab as oryx/127.21.235.66@KRBTEST.COM (short username oryx)
I20260809 12:57:11.087815 31248 webserver.cc:460] Webserver started at http://127.21.235.66:36279/ using document root <none> and password file <none>
I20260809 12:57:11.088502 31248 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:11.088614 31248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:11.088948 31248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:11.091897 31248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "bd14937965824ea7a81d2c51b706a77f"
format_stamp: "Formatted at 2026-08-09 12:57:11 on dist-test-slave-gp6n"
server_key: "9d6896d8453aae39e5aeee8a49125d01"
server_key_iv: "5a4db1b462f46d9463491e47645900ee"
server_key_version: "encryptionkey@0"
I20260809 12:57:11.092700 31248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "bd14937965824ea7a81d2c51b706a77f"
format_stamp: "Formatted at 2026-08-09 12:57:11 on dist-test-slave-gp6n"
server_key: "9d6896d8453aae39e5aeee8a49125d01"
server_key_iv: "5a4db1b462f46d9463491e47645900ee"
server_key_version: "encryptionkey@0"
I20260809 12:57:11.098068 31248 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 12:57:11.101576 31265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:11.102973 31248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 12:57:11.103143 31248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "bd14937965824ea7a81d2c51b706a77f"
format_stamp: "Formatted at 2026-08-09 12:57:11 on dist-test-slave-gp6n"
server_key: "9d6896d8453aae39e5aeee8a49125d01"
server_key_iv: "5a4db1b462f46d9463491e47645900ee"
server_key_version: "encryptionkey@0"
I20260809 12:57:11.103363 31248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:11.128358 31248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:11.132876 31248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:11.133157 31248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:11.135887 31248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:11.136011 31248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:11.136206 31248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:11.136297 31248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:57:11 dist-test-slave-gp6n krb5kdc[30960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280231, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM
I20260809 12:57:11.160234 31116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:11.136299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:60109 (local address 127.21.235.126:43003)
0809 12:57:11.136526 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:11.136535 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:11.138073 (+ 1538us) server_negotiation.cc:407] Connection header received
0809 12:57:11.140152 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:11.140160 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:11.140279 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:11.140406 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:11.142726 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.143801 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.146372 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.147171 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.147319 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:11.152772 (+ 5453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:11.152790 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:11.152795 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:11.152840 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:11.156457 (+ 3617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:11.157527 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:11.157531 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:11.157533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:11.157676 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:11.158323 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:11.158331 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:11.158335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:11.158742 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:11.158923 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:11.159650 (+ 727us) server_negotiation.cc:299] Negotiation successful
0809 12:57:11.159885 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 12:57:11.161302 31276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:11.136979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:43003 (local address 127.21.235.66:60109)
0809 12:57:11.137874 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:11.137925 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:11.139858 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:11.140685 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:11.140696 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:11.141482 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:11.142458 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:11.142498 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.143986 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.143997 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:11.146160 (+ 2163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:11.146182 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.147449 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.147463 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:11.147857 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:11.148710 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:11.148750 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:11.152495 (+ 3745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:11.156723 (+ 4228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:11.156733 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:11.156755 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:11.157351 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:11.157810 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:11.157819 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:11.157826 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:11.158180 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:11.158889 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:11.158896 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:11.159260 (+ 364us) client_negotiation.cc:769] Sending connection context
0809 12:57:11.159535 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 12:57:11.160064 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":388,"threads_started":1}
I20260809 12:57:11.182370 31116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:11.167291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:42239 (local address 127.21.235.126:43003)
0809 12:57:11.167483 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:11.167489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:11.168172 (+ 683us) server_negotiation.cc:407] Connection header received
0809 12:57:11.168350 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:11.168356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:11.168428 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:11.168524 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:11.169466 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.170190 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.172165 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.172722 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.172861 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:11.175663 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:11.175667 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:11.175671 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:11.175708 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:11.179136 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:11.180113 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:11.180121 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:11.180128 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:11.180313 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:11.180732 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:11.180739 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:11.180742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:11.181201 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:11.181386 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:11.181786 (+ 400us) server_negotiation.cc:299] Negotiation successful
0809 12:57:11.182008 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 12:57:11.182873 31354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:11.167392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:43003 (local address 127.21.235.66:42239)
0809 12:57:11.168077 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:11.168093 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:11.168237 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:11.168548 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:11.168552 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:11.169035 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:11.169328 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:11.169338 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.170322 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.170331 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:11.172023 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:11.172035 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.173104 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.173114 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:11.173347 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:11.174060 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:11.174090 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:11.175427 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:11.179331 (+ 3904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:11.179338 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:11.179344 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:11.179973 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:11.180455 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:11.180465 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:11.180468 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:11.180593 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:11.181334 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:11.181340 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:11.181542 (+ 202us) client_negotiation.cc:769] Sending connection context
0809 12:57:11.181694 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 12:57:11.182103 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":389,"threads_started":1}
I20260809 12:57:11.185431 31248 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:43139
I20260809 12:57:11.185473 31397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:43139 every 8 connection(s)
I20260809 12:57:11.186779 31248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:57:11.187383 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 31248
I20260809 12:57:11.187595 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:57:11.188164 22445 external_mini_cluster.cc:1428] Setting key b742bcf26f108413cf84c4a06338772b
I20260809 12:57:11.188892 31398 heartbeater.cc:347] Connected to a master server at 127.21.235.126:43003
I20260809 12:57:11.189143 31398 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:11.189913 31398 heartbeater.cc:511] Master 127.21.235.126:43003 requested a full tablet report, sending...
I20260809 12:57:11.191359 31009 ts_manager.cc:194] Registered new tserver with Master: bd14937965824ea7a81d2c51b706a77f (127.21.235.66:43139)
I20260809 12:57:11.192162 31009 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:60109
WARNING: no policy specified for oryx/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.67.keytab.
Entry for principal oryx/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.67.keytab.
I20260809 12:57:11.245898 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:43003
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.67.keytab
--principal=oryx/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:57:11.598328 31402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:11.598662 31402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:11.598845 31402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:11.606173 31402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:11.606268 31402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:11.606354 31402 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:57:11.606494 31402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:57:11.616077 31402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.67.keytab
--principal=oryx/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.235.126:43003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:11.617187 31402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:11.618458 31402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:11.632568 31408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:11.632539 31409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:11.632629 31411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:11.633293 31402 server_base.cc:1034] running on GCE node
I20260809 12:57:11.633939 31402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:11.634907 31402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:11.636147 31402 hybrid_clock.cc:648] HybridClock initialized: now 1786280231636078 us; error 55 us; skew 500 ppm
I20260809 12:57:11.636439 31402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:11 dist-test-slave-gp6n krb5kdc[30960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280231, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:11.639649 31402 init.cc:377] Logged in from keytab as oryx/127.21.235.67@KRBTEST.COM (short username oryx)
I20260809 12:57:11.641006 31402 webserver.cc:460] Webserver started at http://127.21.235.67:42059/ using document root <none> and password file <none>
I20260809 12:57:11.641691 31402 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:11.641820 31402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:11.642155 31402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:11.645076 31402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4"
format_stamp: "Formatted at 2026-08-09 12:57:11 on dist-test-slave-gp6n"
server_key: "2022ce11e272e418f84954630d5c2ab2"
server_key_iv: "75f542a85a5838d7e6a94f0b2ba43c67"
server_key_version: "encryptionkey@0"
I20260809 12:57:11.645886 31402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4"
format_stamp: "Formatted at 2026-08-09 12:57:11 on dist-test-slave-gp6n"
server_key: "2022ce11e272e418f84954630d5c2ab2"
server_key_iv: "75f542a85a5838d7e6a94f0b2ba43c67"
server_key_version: "encryptionkey@0"
I20260809 12:57:11.651083 31402 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:57:11.654629 31421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:11.655982 31402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:57:11.656132 31402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4"
format_stamp: "Formatted at 2026-08-09 12:57:11 on dist-test-slave-gp6n"
server_key: "2022ce11e272e418f84954630d5c2ab2"
server_key_iv: "75f542a85a5838d7e6a94f0b2ba43c67"
server_key_version: "encryptionkey@0"
I20260809 12:57:11.656342 31402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:11.665256 31243 heartbeater.cc:503] Master 127.21.235.126:43003 was elected leader, sending a full tablet report...
I20260809 12:57:11.668627 31402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:11.672729 31402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:11.672959 31402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:11.675686 31402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:11.675802 31402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:11.676002 31402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:11.676079 31402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:57:11 dist-test-slave-gp6n krb5kdc[30960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280231, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM
I20260809 12:57:11.703491 31430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:11.676693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:43003 (local address 127.21.235.67:48973)
0809 12:57:11.678905 (+ 2212us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:11.678943 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:11.681108 (+ 2165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:11.682274 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:11.682289 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:11.683092 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:11.683844 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:11.683864 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.685759 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.685762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:11.687641 (+ 1879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:11.687653 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.689018 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.689026 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:11.689218 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:11.689992 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:11.690036 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:11.693378 (+ 3342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:11.698011 (+ 4633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:11.698019 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:11.698035 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:11.698842 (+ 807us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:11.699363 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:11.699370 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:11.699377 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:11.699616 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:11.700340 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:11.700348 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:11.700903 (+ 555us) client_negotiation.cc:769] Sending connection context
0809 12:57:11.701209 (+ 306us) client_negotiation.cc:240] Negotiation successful
0809 12:57:11.701821 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1968,"thread_start_us":1708,"threads_started":1}
I20260809 12:57:11.703544 31116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:11.676091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:48973 (local address 127.21.235.126:43003)
0809 12:57:11.676256 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:11.676260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:11.679089 (+ 2829us) server_negotiation.cc:407] Connection header received
0809 12:57:11.681498 (+ 2409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:11.681505 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:11.681604 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:11.681685 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:11.684035 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.685582 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.687906 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.688738 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.688893 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:11.693647 (+ 4754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:11.693653 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:11.693658 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:11.693701 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:11.697854 (+ 4153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:11.699001 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:11.699009 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:11.699015 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:11.699217 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:11.699751 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:11.699756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:11.699759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:11.700192 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:11.700353 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:11.702704 (+ 2351us) server_negotiation.cc:299] Negotiation successful
0809 12:57:11.703005 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 12:57:11.726727 31116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:11.709066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:37487 (local address 127.21.235.126:43003)
0809 12:57:11.709237 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:11.709242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:11.710099 (+ 857us) server_negotiation.cc:407] Connection header received
0809 12:57:11.710384 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:11.710389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:11.710540 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:11.710665 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:11.712543 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.713495 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.716013 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.716680 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.716844 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:11.719820 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:11.719831 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:11.719834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:11.719889 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:11.723803 (+ 3914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:11.724548 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:11.724554 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:11.724558 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:11.724699 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:11.725102 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:11.725108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:11.725112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:11.725412 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:11.725588 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:11.726033 (+ 445us) server_negotiation.cc:299] Negotiation successful
0809 12:57:11.726283 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 12:57:11.726972 31495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:11.709126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:43003 (local address 127.21.235.67:37487)
0809 12:57:11.709988 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:11.710011 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:11.710205 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:11.710895 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:11.710903 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:11.711818 (+ 915us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:11.712366 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:11.712383 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.713652 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.713667 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:11.715838 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:11.715853 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.717126 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.717130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:11.717306 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:11.718211 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:11.718245 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:11.719584 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:11.723956 (+ 4372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:11.723962 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:11.723966 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:11.724414 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:11.724831 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:11.724837 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:11.724841 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:11.724986 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:11.725541 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:11.725549 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:11.725754 (+ 205us) client_negotiation.cc:769] Sending connection context
0809 12:57:11.725925 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 12:57:11.726308 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":443,"threads_started":1}
I20260809 12:57:11.735355 31402 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:41429
I20260809 12:57:11.735440 31560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:41429 every 8 connection(s)
I20260809 12:57:11.736598 31402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:57:11.738631 31561 heartbeater.cc:347] Connected to a master server at 127.21.235.126:43003
I20260809 12:57:11.738870 31561 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:11.739526 31561 heartbeater.cc:511] Master 127.21.235.126:43003 requested a full tablet report, sending...
I20260809 12:57:11.740828 31009 ts_manager.cc:194] Registered new tserver with Master: a6c0f9dc1e994b2d96ab0b3a25f66ef4 (127.21.235.67:41429)
I20260809 12:57:11.741617 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 31402
I20260809 12:57:11.741657 31009 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:48973
I20260809 12:57:11.741791 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:57:11.742273 22445 external_mini_cluster.cc:1428] Setting key 0a08e43bc858ce32d2637e4927760098
I20260809 12:57:11.745129 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:57:11 dist-test-slave-gp6n krb5kdc[30960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280231, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 12:57:11 dist-test-slave-gp6n krb5kdc[30960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:11 dist-test-slave-gp6n krb5kdc[30960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 12:57:11.801064 31116 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:11.789935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43126 (local address 127.21.235.126:43003)
0809 12:57:11.790123 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:11.790127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:11.790388 (+ 261us) server_negotiation.cc:407] Connection header received
0809 12:57:11.790610 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:11.790614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:11.790681 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:11.790779 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:11.792127 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.792845 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.794066 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.794544 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.794636 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:11.800719 (+ 6083us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43126: BlockingRecv error: recv got EOF from 127.0.0.1:43126 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 12:57:11 dist-test-slave-gp6n krb5kdc[30960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280231, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM
I20260809 12:57:11.819190 31116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:11.804260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43134 (local address 127.21.235.126:43003)
0809 12:57:11.804499 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:11.804506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:11.804855 (+ 349us) server_negotiation.cc:407] Connection header received
0809 12:57:11.805068 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:11.805073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:11.805154 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:11.805268 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:11.806643 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.807552 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.808939 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.809419 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.809590 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:11.813856 (+ 4266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:11.813861 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:11.813865 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:11.813900 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:11.816534 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:11.817228 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:11.817235 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:11.817239 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:11.817373 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:11.817749 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:11.817755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:11.817766 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:11.818045 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:11.818350 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:11.818644 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 12:57:11.818881 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 12:57:11.823313 31009 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43134:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:57:11.830528 31009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:57:11.867187 31584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:11.842924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60100 (local address 127.21.235.66:43139)
0809 12:57:11.855245 (+ 12321us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:11.855258 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:11.855297 (+ 39us) server_negotiation.cc:407] Connection header received
0809 12:57:11.855439 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:11.855451 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:11.855803 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:11.857265 (+ 1462us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:11.857519 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.859369 (+ 1850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.864177 (+ 4808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.865680 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.865824 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:11.866060 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:11.866690 (+ 630us) server_negotiation.cc:299] Negotiation successful
0809 12:57:11.866883 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12085,"thread_start_us":13088,"threads_started":1}
I20260809 12:57:11.867228 31583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:11.842721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:43139 (local address 127.0.0.1:60100)
0809 12:57:11.844643 (+ 1922us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:11.844688 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:11.844857 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:11.856380 (+ 11523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:11.856394 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:11.856440 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:11.856983 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:11.857007 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.860504 (+ 3497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.860514 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:11.863866 (+ 3352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:11.863890 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.866188 (+ 2298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.866195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:11.866395 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:11.866486 (+ 91us) client_negotiation.cc:769] Sending connection context
0809 12:57:11.866619 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 12:57:11.866807 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1641,"thread_start_us":17058,"threads_started":1}
I20260809 12:57:11.871461 31581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:11.842856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:41429 (local address 127.0.0.1:35076)
0809 12:57:11.860122 (+ 17266us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:11.860150 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:11.860304 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:11.863994 (+ 3690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:11.863999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:11.864039 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:11.864486 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:11.864498 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.867094 (+ 2596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.867100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:11.869041 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:11.869053 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.870654 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.870661 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:11.870821 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:11.870855 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 12:57:11.870955 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 12:57:11.871082 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17098,"mutex_wait_us":1,"thread_start_us":1385,"threads_started":1}
I20260809 12:57:11.871524 31585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:11.843595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35076 (local address 127.21.235.67:41429)
0809 12:57:11.862510 (+ 18915us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:11.862523 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:11.862568 (+ 45us) server_negotiation.cc:407] Connection header received
0809 12:57:11.862692 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:11.862703 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:11.863050 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:11.863304 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:11.864723 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.866946 (+ 2223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.869198 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.870454 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.870585 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:11.870808 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:11.871012 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 12:57:11.871181 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18634,"thread_start_us":20043,"threads_started":1}
I20260809 12:57:11.871487 31320 tablet_service.cc:1467] Processing CreateTablet for tablet 06f0b05d58354bffb9eaa17feae90245 (DEFAULT_TABLE table=test-table [id=114d6c640ee14920ac5ed578528ce3a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:11.872557 31586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:11.845055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:42239 (local address 127.0.0.1:37914)
0809 12:57:11.864315 (+ 19260us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:11.864331 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:11.864602 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:11.865189 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:11.865194 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:11.865219 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:11.865620 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:11.865635 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.867810 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.867816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:11.869676 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:11.869692 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.871833 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.871839 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:11.871989 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:11.872012 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 12:57:11.872115 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 12:57:11.872239 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19098,"thread_start_us":18876,"threads_started":1}
I20260809 12:57:11.872947 31587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:11.845505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37914 (local address 127.21.235.65:42239)
0809 12:57:11.863741 (+ 18236us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:11.863751 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:11.864466 (+ 715us) server_negotiation.cc:407] Connection header received
0809 12:57:11.864768 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:11.864774 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:11.865005 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:11.865204 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:11.865847 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.867621 (+ 1774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.869876 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:11.871315 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:11.871467 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:11.871681 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:11.872157 (+ 476us) server_negotiation.cc:299] Negotiation successful
0809 12:57:11.872409 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18029,"thread_start_us":17782,"threads_started":1}
I20260809 12:57:11.873737 31320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06f0b05d58354bffb9eaa17feae90245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:11.875361 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 06f0b05d58354bffb9eaa17feae90245 (DEFAULT_TABLE table=test-table [id=114d6c640ee14920ac5ed578528ce3a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:11.876564 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06f0b05d58354bffb9eaa17feae90245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:11.876504 31166 tablet_service.cc:1467] Processing CreateTablet for tablet 06f0b05d58354bffb9eaa17feae90245 (DEFAULT_TABLE table=test-table [id=114d6c640ee14920ac5ed578528ce3a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:11.877875 31166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06f0b05d58354bffb9eaa17feae90245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:11.889086 31588 tablet_bootstrap.cc:492] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Bootstrap starting.
I20260809 12:57:11.890781 31591 tablet_bootstrap.cc:492] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f: Bootstrap starting.
I20260809 12:57:11.892525 31592 tablet_bootstrap.cc:492] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f: Bootstrap starting.
I20260809 12:57:11.894345 31588 tablet_bootstrap.cc:654] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:11.894351 31591 tablet_bootstrap.cc:654] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:11.895375 31592 tablet_bootstrap.cc:654] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:11.895675 31591 log.cc:824] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f: Log is configured to *not* fsync() on all Append() calls
I20260809 12:57:11.895675 31588 log.cc:824] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Log is configured to *not* fsync() on all Append() calls
I20260809 12:57:11.896507 31592 log.cc:824] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f: Log is configured to *not* fsync() on all Append() calls
I20260809 12:57:11.898747 31591 tablet_bootstrap.cc:492] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f: No bootstrap required, opened a new log
I20260809 12:57:11.898857 31588 tablet_bootstrap.cc:492] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: No bootstrap required, opened a new log
I20260809 12:57:11.899070 31591 ts_tablet_manager.cc:1397] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 12:57:11.899150 31588 ts_tablet_manager.cc:1397] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 12:57:11.899739 31592 tablet_bootstrap.cc:492] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f: No bootstrap required, opened a new log
I20260809 12:57:11.899992 31592 ts_tablet_manager.cc:1397] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 12:57:11.907490 31591 raft_consensus.cc:348] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } }
I20260809 12:57:11.907879 31591 raft_consensus.cc:374] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:11.907977 31591 raft_consensus.cc:729] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd14937965824ea7a81d2c51b706a77f, State: Initialized, Role: FOLLOWER
I20260809 12:57:11.908582 31591 consensus_queue.cc:260] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } }
I20260809 12:57:11.909655 31398 heartbeater.cc:503] Master 127.21.235.126:43003 was elected leader, sending a full tablet report...
I20260809 12:57:11.910344 31591 ts_tablet_manager.cc:1428] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260809 12:57:11.910118 31588 raft_consensus.cc:348] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } }
I20260809 12:57:11.910576 31588 raft_consensus.cc:374] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:11.910681 31588 raft_consensus.cc:729] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6c0f9dc1e994b2d96ab0b3a25f66ef4, State: Initialized, Role: FOLLOWER
I20260809 12:57:11.910718 31592 raft_consensus.cc:348] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } }
I20260809 12:57:11.911101 31592 raft_consensus.cc:374] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:11.911204 31592 raft_consensus.cc:729] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b16ecc6081944b688b42d8afb5a861f, State: Initialized, Role: FOLLOWER
I20260809 12:57:11.911270 31588 consensus_queue.cc:260] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } }
I20260809 12:57:11.911798 31592 consensus_queue.cc:260] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } }
I20260809 12:57:11.912333 31561 heartbeater.cc:503] Master 127.21.235.126:43003 was elected leader, sending a full tablet report...
I20260809 12:57:11.913208 31588 ts_tablet_manager.cc:1428] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260809 12:57:11.915545 31592 ts_tablet_manager.cc:1428] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
W20260809 12:57:11.938815 31399 tablet.cc:2369] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:57:11.988189 31562 tablet.cc:2369] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:57:12.104605 31598 raft_consensus.cc:482] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:57:12.104939 31598 raft_consensus.cc:504] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } }
I20260809 12:57:12.106982 31598 leader_election.cc:288] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd14937965824ea7a81d2c51b706a77f (127.21.235.66:43139), a6c0f9dc1e994b2d96ab0b3a25f66ef4 (127.21.235.67:41429)
I20260809 12:57:12.116353 31585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:12.107462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:39563 (local address 127.21.235.67:41429)
0809 12:57:12.107803 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:12.107811 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:12.108338 (+ 527us) server_negotiation.cc:407] Connection header received
0809 12:57:12.108584 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:12.108591 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:12.108671 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:12.108768 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:12.109534 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.110677 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.113944 (+ 3267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.115223 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.115399 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:12.115475 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:12.115952 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 12:57:12.116090 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260809 12:57:12.116390 31584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:12.107368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:51325 (local address 127.21.235.66:43139)
0809 12:57:12.107593 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:12.107602 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:12.108442 (+ 840us) server_negotiation.cc:407] Connection header received
0809 12:57:12.108661 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:12.108666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:12.108749 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:12.108840 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:12.109534 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.110681 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.113953 (+ 3272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.115223 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.115398 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:12.115478 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:12.115952 (+ 474us) server_negotiation.cc:299] Negotiation successful
0809 12:57:12.116092 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 12:57:12.116531 31599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:12.107372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:43139 (local address 127.21.235.65:51325)
0809 12:57:12.108354 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:12.108368 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:12.108479 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:12.108859 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:12.108862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:12.108883 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:12.109381 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:12.109389 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.110807 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.110810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:12.113773 (+ 2963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:12.113789 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.115503 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.115509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:12.115717 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:12.115789 (+ 72us) client_negotiation.cc:769] Sending connection context
0809 12:57:12.115888 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 12:57:12.116037 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"mutex_wait_us":299,"thread_start_us":417,"threads_started":1}
I20260809 12:57:12.116799 31600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:12.107303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:41429 (local address 127.21.235.65:39563)
0809 12:57:12.108234 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:12.108259 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:12.108416 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:12.108793 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:12.108796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:12.108847 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:12.109348 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:12.109363 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.110805 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.110810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:12.113778 (+ 2968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:12.113788 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.115483 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.115490 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:12.115726 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:12.115797 (+ 71us) client_negotiation.cc:769] Sending connection context
0809 12:57:12.115888 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 12:57:12.116037 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"mutex_wait_us":332,"thread_start_us":762,"threads_started":1}
I20260809 12:57:12.117690 31342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06f0b05d58354bffb9eaa17feae90245" candidate_uuid: "1b16ecc6081944b688b42d8afb5a861f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd14937965824ea7a81d2c51b706a77f" is_pre_election: true
I20260809 12:57:12.117733 31507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06f0b05d58354bffb9eaa17feae90245" candidate_uuid: "1b16ecc6081944b688b42d8afb5a861f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" is_pre_election: true
I20260809 12:57:12.118301 31507 raft_consensus.cc:2454] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b16ecc6081944b688b42d8afb5a861f in term 0.
I20260809 12:57:12.118314 31342 raft_consensus.cc:2454] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b16ecc6081944b688b42d8afb5a861f in term 0.
I20260809 12:57:12.119011 31110 leader_election.cc:302] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b16ecc6081944b688b42d8afb5a861f, bd14937965824ea7a81d2c51b706a77f; no voters:
I20260809 12:57:12.119366 31598 raft_consensus.cc:2781] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:57:12.119519 31598 raft_consensus.cc:482] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:57:12.119643 31598 raft_consensus.cc:3037] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:12.121940 31598 raft_consensus.cc:504] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } }
I20260809 12:57:12.122562 31598 leader_election.cc:288] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [CANDIDATE]: Term 1 election: Requested vote from peers bd14937965824ea7a81d2c51b706a77f (127.21.235.66:43139), a6c0f9dc1e994b2d96ab0b3a25f66ef4 (127.21.235.67:41429)
I20260809 12:57:12.123030 31507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06f0b05d58354bffb9eaa17feae90245" candidate_uuid: "1b16ecc6081944b688b42d8afb5a861f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4"
I20260809 12:57:12.123050 31342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06f0b05d58354bffb9eaa17feae90245" candidate_uuid: "1b16ecc6081944b688b42d8afb5a861f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd14937965824ea7a81d2c51b706a77f"
I20260809 12:57:12.123230 31507 raft_consensus.cc:3037] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:12.123251 31342 raft_consensus.cc:3037] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:12.125402 31507 raft_consensus.cc:2454] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b16ecc6081944b688b42d8afb5a861f in term 1.
I20260809 12:57:12.125494 31342 raft_consensus.cc:2454] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b16ecc6081944b688b42d8afb5a861f in term 1.
I20260809 12:57:12.125837 31109 leader_election.cc:302] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b16ecc6081944b688b42d8afb5a861f, a6c0f9dc1e994b2d96ab0b3a25f66ef4; no voters:
I20260809 12:57:12.126137 31598 raft_consensus.cc:2781] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:57:12.126601 31598 raft_consensus.cc:686] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [term 1 LEADER]: Becoming Leader. State: Replica: 1b16ecc6081944b688b42d8afb5a861f, State: Running, Role: LEADER
I20260809 12:57:12.127315 31598 consensus_queue.cc:237] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } }
I20260809 12:57:12.131773 31007 catalog_manager.cc:5495] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b16ecc6081944b688b42d8afb5a861f (127.21.235.65). New cstate: current_term: 1 leader_uuid: "1b16ecc6081944b688b42d8afb5a861f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } health_report { overall_health: HEALTHY } } }
I20260809 12:57:12.148316 31587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:12.142606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37916 (local address 127.21.235.65:42239)
0809 12:57:12.142816 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:12.142823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:12.142947 (+ 124us) server_negotiation.cc:407] Connection header received
0809 12:57:12.143169 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:12.143172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:12.143261 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:12.143403 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:57:12.143775 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.144623 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.146183 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.146681 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.146813 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:12.147242 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:57:12.147763 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:57:12.147920 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:12.147974 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 12:57:12.148056 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:57:12.156277 31342 raft_consensus.cc:1261] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f [term 1 FOLLOWER]: Refusing update from remote peer 1b16ecc6081944b688b42d8afb5a861f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:57:12.156279 31507 raft_consensus.cc:1261] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 FOLLOWER]: Refusing update from remote peer 1b16ecc6081944b688b42d8afb5a861f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 12:57:12.157512 31245 tablet.cc:2369] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:57:12.157424 31601 consensus_queue.cc:1035] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:57:12.157892 31598 consensus_queue.cc:1035] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:57:12.176091 31605 mvcc.cc:205] Tried to move back new op lower bound from 7316603830902153216 to 7316603830797180928. Current Snapshot: MvccSnapshot[applied={T|T < 7316603830902153216}]
I20260809 12:57:12.176535 31606 mvcc.cc:205] Tried to move back new op lower bound from 7316603830902153216 to 7316603830797180928. Current Snapshot: MvccSnapshot[applied={T|T < 7316603830902153216}]
I20260809 12:57:12.188428 31009 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:57:12.188750 31009 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:57:12.192808 31009 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99: Sending DeleteTablet for 3 replicas of tablet 06f0b05d58354bffb9eaa17feae90245
I20260809 12:57:12.193984 31320 tablet_service.cc:1514] Processing DeleteTablet for tablet 06f0b05d58354bffb9eaa17feae90245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:12 UTC) from {username='oryx', principal='oryx/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:60100
I20260809 12:57:12.194315 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet 06f0b05d58354bffb9eaa17feae90245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:12 UTC) from {username='oryx', principal='oryx/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:35076
I20260809 12:57:12.195127 31166 tablet_service.cc:1514] Processing DeleteTablet for tablet 06f0b05d58354bffb9eaa17feae90245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:12 UTC) from {username='oryx', principal='oryx/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:37914
I20260809 12:57:12.195180 31009 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43134:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:57:12.195816 31009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:57:12.204640 31474 tablet_service.cc:1467] Processing CreateTablet for tablet a7210e2b7e5c44a8b28af4f88a644962 (DEFAULT_TABLE table=test-table [id=bc2d8dea17bb41e18bfec6d3fc8fca25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:12.205185 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7210e2b7e5c44a8b28af4f88a644962. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:12.205711 31319 tablet_service.cc:1467] Processing CreateTablet for tablet a7210e2b7e5c44a8b28af4f88a644962 (DEFAULT_TABLE table=test-table [id=bc2d8dea17bb41e18bfec6d3fc8fca25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:12.206427 31319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7210e2b7e5c44a8b28af4f88a644962. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:12.206977 31165 tablet_service.cc:1467] Processing CreateTablet for tablet a7210e2b7e5c44a8b28af4f88a644962 (DEFAULT_TABLE table=test-table [id=bc2d8dea17bb41e18bfec6d3fc8fca25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:12.207571 31165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7210e2b7e5c44a8b28af4f88a644962. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:12.212828 31588 tablet_bootstrap.cc:492] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Bootstrap starting.
I20260809 12:57:12.214738 31588 tablet_bootstrap.cc:654] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:12.222111 31592 tablet_bootstrap.cc:492] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f: Bootstrap starting.
I20260809 12:57:12.223842 31592 tablet_bootstrap.cc:654] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:12.228037 31591 tablet_bootstrap.cc:492] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f: Bootstrap starting.
I20260809 12:57:12.230180 31591 tablet_bootstrap.cc:654] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:12.236941 31588 tablet_bootstrap.cc:492] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: No bootstrap required, opened a new log
I20260809 12:57:12.237149 31588 ts_tablet_manager.cc:1397] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.007s
I20260809 12:57:12.238274 31588 raft_consensus.cc:348] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.238626 31588 raft_consensus.cc:374] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:12.238721 31588 raft_consensus.cc:729] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6c0f9dc1e994b2d96ab0b3a25f66ef4, State: Initialized, Role: FOLLOWER
I20260809 12:57:12.238871 31591 tablet_bootstrap.cc:492] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f: No bootstrap required, opened a new log
I20260809 12:57:12.239110 31588 consensus_queue.cc:260] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.239620 31591 ts_tablet_manager.cc:1397] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20260809 12:57:12.240123 31615 tablet_replica.cc:330] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f: stopping tablet replica
I20260809 12:57:12.240697 31615 raft_consensus.cc:2229] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:57:12.240621 31591 raft_consensus.cc:348] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.240896 31591 raft_consensus.cc:374] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:12.241022 31591 raft_consensus.cc:729] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd14937965824ea7a81d2c51b706a77f, State: Initialized, Role: FOLLOWER
I20260809 12:57:12.241447 31614 tablet_replica.cc:330] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: stopping tablet replica
I20260809 12:57:12.241355 31591 consensus_queue.cc:260] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.241986 31591 ts_tablet_manager.cc:1428] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 12:57:12.242443 31614 raft_consensus.cc:2229] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:57:12.242635 31592 tablet_bootstrap.cc:492] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f: No bootstrap required, opened a new log
I20260809 12:57:12.242789 31592 ts_tablet_manager.cc:1397] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.003s
I20260809 12:57:12.243247 31615 raft_consensus.cc:2258] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:12.243721 31614 raft_consensus.cc:2258] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:12.243664 31592 raft_consensus.cc:348] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.243957 31592 raft_consensus.cc:374] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:12.244071 31592 raft_consensus.cc:729] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b16ecc6081944b688b42d8afb5a861f, State: Initialized, Role: FOLLOWER
I20260809 12:57:12.244329 31592 consensus_queue.cc:260] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.244946 31592 ts_tablet_manager.cc:1428] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 12:57:12.246515 31588 ts_tablet_manager.cc:1428] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260809 12:57:12.246807 31613 tablet_replica.cc:330] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f: stopping tablet replica
I20260809 12:57:12.247272 31613 raft_consensus.cc:2229] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:57:12.247871 31613 raft_consensus.cc:2258] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:12.253456 31615 ts_tablet_manager.cc:1905] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:12.258620 31613 ts_tablet_manager.cc:1905] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:12.260623 31614 ts_tablet_manager.cc:1905] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:12.262612 31615 ts_tablet_manager.cc:1918] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:57:12.262871 31615 log.cc:1196] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/06f0b05d58354bffb9eaa17feae90245
I20260809 12:57:12.263582 31615 ts_tablet_manager.cc:1939] T 06f0b05d58354bffb9eaa17feae90245 P 1b16ecc6081944b688b42d8afb5a861f: Deleting consensus metadata
I20260809 12:57:12.265504 30992 catalog_manager.cc:4841] TS 1b16ecc6081944b688b42d8afb5a861f (127.21.235.65:42239): tablet 06f0b05d58354bffb9eaa17feae90245 (table test-table [id=114d6c640ee14920ac5ed578528ce3a1]) successfully deleted
I20260809 12:57:12.266219 31613 ts_tablet_manager.cc:1918] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:57:12.266384 31613 log.cc:1196] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/06f0b05d58354bffb9eaa17feae90245
I20260809 12:57:12.266938 31613 ts_tablet_manager.cc:1939] T 06f0b05d58354bffb9eaa17feae90245 P bd14937965824ea7a81d2c51b706a77f: Deleting consensus metadata
I20260809 12:57:12.268191 31614 ts_tablet_manager.cc:1918] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:57:12.268355 31614 log.cc:1196] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/06f0b05d58354bffb9eaa17feae90245
I20260809 12:57:12.268602 30994 catalog_manager.cc:4841] TS bd14937965824ea7a81d2c51b706a77f (127.21.235.66:43139): tablet 06f0b05d58354bffb9eaa17feae90245 (table test-table [id=114d6c640ee14920ac5ed578528ce3a1]) successfully deleted
I20260809 12:57:12.269132 31614 ts_tablet_manager.cc:1939] T 06f0b05d58354bffb9eaa17feae90245 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Deleting consensus metadata
I20260809 12:57:12.270444 30992 catalog_manager.cc:4841] TS a6c0f9dc1e994b2d96ab0b3a25f66ef4 (127.21.235.67:41429): tablet 06f0b05d58354bffb9eaa17feae90245 (table test-table [id=114d6c640ee14920ac5ed578528ce3a1]) successfully deleted
I20260809 12:57:12.289942 31598 raft_consensus.cc:482] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:57:12.290153 31598 raft_consensus.cc:504] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.290920 31598 leader_election.cc:288] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd14937965824ea7a81d2c51b706a77f (127.21.235.66:43139), a6c0f9dc1e994b2d96ab0b3a25f66ef4 (127.21.235.67:41429)
I20260809 12:57:12.291424 31340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7210e2b7e5c44a8b28af4f88a644962" candidate_uuid: "1b16ecc6081944b688b42d8afb5a861f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd14937965824ea7a81d2c51b706a77f" is_pre_election: true
I20260809 12:57:12.291587 31507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7210e2b7e5c44a8b28af4f88a644962" candidate_uuid: "1b16ecc6081944b688b42d8afb5a861f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" is_pre_election: true
I20260809 12:57:12.291739 31340 raft_consensus.cc:2454] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b16ecc6081944b688b42d8afb5a861f in term 0.
I20260809 12:57:12.291853 31507 raft_consensus.cc:2454] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b16ecc6081944b688b42d8afb5a861f in term 0.
I20260809 12:57:12.292214 31110 leader_election.cc:302] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b16ecc6081944b688b42d8afb5a861f, bd14937965824ea7a81d2c51b706a77f; no voters:
I20260809 12:57:12.292502 31598 raft_consensus.cc:2781] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:57:12.292635 31598 raft_consensus.cc:482] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:57:12.292754 31598 raft_consensus.cc:3037] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:12.295049 31598 raft_consensus.cc:504] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.295608 31598 leader_election.cc:288] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [CANDIDATE]: Term 1 election: Requested vote from peers bd14937965824ea7a81d2c51b706a77f (127.21.235.66:43139), a6c0f9dc1e994b2d96ab0b3a25f66ef4 (127.21.235.67:41429)
I20260809 12:57:12.296078 31340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7210e2b7e5c44a8b28af4f88a644962" candidate_uuid: "1b16ecc6081944b688b42d8afb5a861f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd14937965824ea7a81d2c51b706a77f"
I20260809 12:57:12.296115 31507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7210e2b7e5c44a8b28af4f88a644962" candidate_uuid: "1b16ecc6081944b688b42d8afb5a861f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4"
I20260809 12:57:12.296298 31340 raft_consensus.cc:3037] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:12.296299 31507 raft_consensus.cc:3037] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:12.299244 31340 raft_consensus.cc:2454] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b16ecc6081944b688b42d8afb5a861f in term 1.
I20260809 12:57:12.299244 31507 raft_consensus.cc:2454] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b16ecc6081944b688b42d8afb5a861f in term 1.
I20260809 12:57:12.299616 31109 leader_election.cc:302] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b16ecc6081944b688b42d8afb5a861f, a6c0f9dc1e994b2d96ab0b3a25f66ef4; no voters:
I20260809 12:57:12.299882 31598 raft_consensus.cc:2781] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:57:12.300052 31598 raft_consensus.cc:686] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [term 1 LEADER]: Becoming Leader. State: Replica: 1b16ecc6081944b688b42d8afb5a861f, State: Running, Role: LEADER
I20260809 12:57:12.300447 31598 consensus_queue.cc:237] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.303521 31009 catalog_manager.cc:5495] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b16ecc6081944b688b42d8afb5a861f (127.21.235.65). New cstate: current_term: 1 leader_uuid: "1b16ecc6081944b688b42d8afb5a861f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } health_report { overall_health: UNKNOWN } } }
Aug 09 12:57:12 dist-test-slave-gp6n krb5kdc[30960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280229, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.126@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM
I20260809 12:57:12.349709 31628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:12.328693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:43003 (local address 127.0.0.1:43144)
0809 12:57:12.329497 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:12.329525 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:12.329703 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:12.330504 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:12.330510 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:12.331483 (+ 973us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:12.331817 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:12.331831 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.332932 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.332939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:12.334533 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:12.334549 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.335245 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.335255 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:12.335433 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:12.340519 (+ 5086us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:12.340544 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:12.343310 (+ 2766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:12.346831 (+ 3521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:12.346845 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:12.346854 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:12.347312 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:12.347717 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:12.347722 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:12.347726 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:12.347884 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:12.348489 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:12.348498 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:12.348678 (+ 180us) client_negotiation.cc:769] Sending connection context
0809 12:57:12.348863 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 12:57:12.349179 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":399,"threads_started":1}
I20260809 12:57:12.349833 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:12.328711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43144 (local address 127.21.235.126:43003)
0809 12:57:12.330198 (+ 1487us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:12.330202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:12.330221 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:57:12.330261 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:12.330264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:12.330316 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:12.330457 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:12.332011 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.332805 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.334669 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.335055 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.335156 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:12.343566 (+ 8410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:12.343571 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:12.343574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:12.343605 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:12.346667 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:12.347444 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:12.347455 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:12.347457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:12.347594 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:12.347998 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:12.348007 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:12.348010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:12.348345 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:12.348511 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:12.348962 (+ 451us) server_negotiation.cc:299] Negotiation successful
0809 12:57:12.349356 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":146,"server-negotiator.queue_time_us":1400,"spinlock_wait_cycles":15616,"thread_start_us":1281,"threads_started":1}
I20260809 12:57:12.355477 31007 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:43144:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 12:57:12.356518 31007 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:57:12.363742 31166 tablet_service.cc:1467] Processing CreateTablet for tablet db135aba68aa4207b82c68b3b188f0c1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5811896a752c46b3a01b3497bc6369b2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:57:12.364118 31320 tablet_service.cc:1467] Processing CreateTablet for tablet db135aba68aa4207b82c68b3b188f0c1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5811896a752c46b3a01b3497bc6369b2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:57:12.364459 31166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db135aba68aa4207b82c68b3b188f0c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:12.364707 31320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db135aba68aa4207b82c68b3b188f0c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:12.370837 31592 tablet_bootstrap.cc:492] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f: Bootstrap starting.
I20260809 12:57:12.371436 31475 tablet_service.cc:1467] Processing CreateTablet for tablet db135aba68aa4207b82c68b3b188f0c1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5811896a752c46b3a01b3497bc6369b2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:57:12.372093 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db135aba68aa4207b82c68b3b188f0c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:12.372817 31592 tablet_bootstrap.cc:654] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:12.377830 31591 tablet_bootstrap.cc:492] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f: Bootstrap starting.
I20260809 12:57:12.379311 31592 tablet_bootstrap.cc:492] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f: No bootstrap required, opened a new log
I20260809 12:57:12.379503 31592 ts_tablet_manager.cc:1397] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.002s
I20260809 12:57:12.379730 31591 tablet_bootstrap.cc:654] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:12.380295 31592 raft_consensus.cc:348] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.380559 31592 raft_consensus.cc:374] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:12.380638 31592 raft_consensus.cc:729] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b16ecc6081944b688b42d8afb5a861f, State: Initialized, Role: FOLLOWER
I20260809 12:57:12.381040 31588 tablet_bootstrap.cc:492] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Bootstrap starting.
I20260809 12:57:12.380924 31592 consensus_queue.cc:260] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.381544 31592 ts_tablet_manager.cc:1428] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 12:57:12.382613 31588 tablet_bootstrap.cc:654] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:12.382648 31598 tablet_replica.cc:439] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } }
I20260809 12:57:12.383037 31598 tablet_replica.cc:442] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:57:12.388603 31598 raft_consensus.cc:482] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:57:12.388808 31598 raft_consensus.cc:504] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.389483 31598 leader_election.cc:288] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd14937965824ea7a81d2c51b706a77f (127.21.235.66:43139), a6c0f9dc1e994b2d96ab0b3a25f66ef4 (127.21.235.67:41429)
I20260809 12:57:12.390154 31340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db135aba68aa4207b82c68b3b188f0c1" candidate_uuid: "1b16ecc6081944b688b42d8afb5a861f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd14937965824ea7a81d2c51b706a77f" is_pre_election: true
I20260809 12:57:12.391306 31507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db135aba68aa4207b82c68b3b188f0c1" candidate_uuid: "1b16ecc6081944b688b42d8afb5a861f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" is_pre_election: true
I20260809 12:57:12.392184 31588 tablet_bootstrap.cc:492] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: No bootstrap required, opened a new log
W20260809 12:57:12.392127 31109 leader_election.cc:341] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6c0f9dc1e994b2d96ab0b3a25f66ef4 (127.21.235.67:41429): Illegal state: must be running to vote when last-logged opid is not known
I20260809 12:57:12.392379 31588 ts_tablet_manager.cc:1397] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
W20260809 12:57:12.392676 31110 leader_election.cc:341] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd14937965824ea7a81d2c51b706a77f (127.21.235.66:43139): Illegal state: must be running to vote when last-logged opid is not known
I20260809 12:57:12.392810 31110 leader_election.cc:302] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b16ecc6081944b688b42d8afb5a861f; no voters: a6c0f9dc1e994b2d96ab0b3a25f66ef4, bd14937965824ea7a81d2c51b706a77f
I20260809 12:57:12.393296 31598 raft_consensus.cc:2726] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 12:57:12.393374 31588 raft_consensus.cc:348] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.393640 31588 raft_consensus.cc:374] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:12.393724 31588 raft_consensus.cc:729] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6c0f9dc1e994b2d96ab0b3a25f66ef4, State: Initialized, Role: FOLLOWER
I20260809 12:57:12.393996 31588 consensus_queue.cc:260] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.394640 31588 ts_tablet_manager.cc:1428] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 12:57:12.394531 31597 tablet_replica.cc:439] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } }
I20260809 12:57:12.394893 31597 tablet_replica.cc:442] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:57:12.395433 31591 tablet_bootstrap.cc:492] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f: No bootstrap required, opened a new log
I20260809 12:57:12.395679 31591 ts_tablet_manager.cc:1397] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.004s
I20260809 12:57:12.396612 31591 raft_consensus.cc:348] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.396955 31591 raft_consensus.cc:374] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:12.397085 31591 raft_consensus.cc:729] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd14937965824ea7a81d2c51b706a77f, State: Initialized, Role: FOLLOWER
I20260809 12:57:12.397487 31591 consensus_queue.cc:260] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.398056 31596 tablet_replica.cc:439] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } }
I20260809 12:57:12.399271 31596 tablet_replica.cc:442] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:57:12.399801 31591 ts_tablet_manager.cc:1428] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260809 12:57:12.648942 31597 raft_consensus.cc:482] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:57:12.649205 31597 raft_consensus.cc:504] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.651027 31597 leader_election.cc:288] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b16ecc6081944b688b42d8afb5a861f (127.21.235.65:42239), bd14937965824ea7a81d2c51b706a77f (127.21.235.66:43139)
I20260809 12:57:12.670526 31637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:12.651349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:42239 (local address 127.21.235.67:43339)
0809 12:57:12.662858 (+ 11509us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:12.662879 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:12.663043 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:12.663834 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:12.663839 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:12.663874 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:12.664121 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:12.664131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.665773 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.665779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:12.667986 (+ 2207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:12.668001 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.669646 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.669653 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:12.669871 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:12.669906 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 12:57:12.669999 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 12:57:12.670141 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11392,"threads_started":1}
I20260809 12:57:12.671285 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:12.651603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:43339 (local address 127.21.235.65:42239)
0809 12:57:12.662513 (+ 10910us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:12.662520 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:12.663219 (+ 699us) server_negotiation.cc:407] Connection header received
0809 12:57:12.663436 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:12.663442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:12.663513 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:12.663641 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:12.664421 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.665599 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.668162 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.669297 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.669472 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:12.669549 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:12.670836 (+ 1287us) server_negotiation.cc:299] Negotiation successful
0809 12:57:12.670975 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10760,"thread_start_us":11516,"threads_started":1}
I20260809 12:57:12.672241 31187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db135aba68aa4207b82c68b3b188f0c1" candidate_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b16ecc6081944b688b42d8afb5a861f" is_pre_election: true
I20260809 12:57:12.672685 31187 raft_consensus.cc:2454] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6c0f9dc1e994b2d96ab0b3a25f66ef4 in term 0.
I20260809 12:57:12.673274 31423 leader_election.cc:302] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b16ecc6081944b688b42d8afb5a861f, a6c0f9dc1e994b2d96ab0b3a25f66ef4; no voters:
I20260809 12:57:12.673683 31597 raft_consensus.cc:2781] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:57:12.673841 31597 raft_consensus.cc:482] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:57:12.673933 31597 raft_consensus.cc:3037] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:12.676697 31597 raft_consensus.cc:504] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.677523 31597 leader_election.cc:288] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [CANDIDATE]: Term 1 election: Requested vote from peers 1b16ecc6081944b688b42d8afb5a861f (127.21.235.65:42239), bd14937965824ea7a81d2c51b706a77f (127.21.235.66:43139)
I20260809 12:57:12.677919 31187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db135aba68aa4207b82c68b3b188f0c1" candidate_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b16ecc6081944b688b42d8afb5a861f"
I20260809 12:57:12.678131 31187 raft_consensus.cc:3037] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:12.678894 31634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:12.651352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:43139 (local address 127.21.235.67:34033)
0809 12:57:12.670650 (+ 19298us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:12.670678 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:12.670826 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:12.671883 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:12.671887 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:12.671911 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:12.672208 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:12.672221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.673792 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.673797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:12.676003 (+ 2206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:12.676018 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.678008 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.678014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:12.678203 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:12.678241 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 12:57:12.678354 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 12:57:12.678541 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19130,"mutex_wait_us":211,"thread_start_us":11059,"threads_started":1}
I20260809 12:57:12.679145 31635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:12.651524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:34033 (local address 127.21.235.66:43139)
0809 12:57:12.671546 (+ 20022us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:12.671552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:12.671574 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:57:12.671652 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:12.671659 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:12.671750 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:12.671869 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:12.672626 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.673655 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.676163 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.677536 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.677810 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:12.677891 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:12.678489 (+ 598us) server_negotiation.cc:299] Negotiation successful
0809 12:57:12.678632 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19906,"thread_start_us":19793,"threads_started":1}
I20260809 12:57:12.679752 31340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db135aba68aa4207b82c68b3b188f0c1" candidate_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd14937965824ea7a81d2c51b706a77f" is_pre_election: true
I20260809 12:57:12.679987 31340 raft_consensus.cc:2454] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6c0f9dc1e994b2d96ab0b3a25f66ef4 in term 0.
I20260809 12:57:12.680501 31340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db135aba68aa4207b82c68b3b188f0c1" candidate_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd14937965824ea7a81d2c51b706a77f"
I20260809 12:57:12.680670 31340 raft_consensus.cc:3037] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:12.680850 31187 raft_consensus.cc:2454] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6c0f9dc1e994b2d96ab0b3a25f66ef4 in term 1.
I20260809 12:57:12.681193 31423 leader_election.cc:302] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b16ecc6081944b688b42d8afb5a861f, a6c0f9dc1e994b2d96ab0b3a25f66ef4; no voters:
I20260809 12:57:12.681501 31597 raft_consensus.cc:2781] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:57:12.682194 31597 raft_consensus.cc:686] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 LEADER]: Becoming Leader. State: Replica: a6c0f9dc1e994b2d96ab0b3a25f66ef4, State: Running, Role: LEADER
I20260809 12:57:12.682864 31597 consensus_queue.cc:237] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:12.683698 31340 raft_consensus.cc:2454] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6c0f9dc1e994b2d96ab0b3a25f66ef4 in term 1.
I20260809 12:57:12.684674 31638 tablet_replica.cc:439] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: TxnStatusTablet state changed. Reason: New leader a6c0f9dc1e994b2d96ab0b3a25f66ef4. Latest consensus state: current_term: 1 leader_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } }
I20260809 12:57:12.685127 31638 tablet_replica.cc:442] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:57:12.685917 31642 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 12:57:12.686131 31642 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 12:57:12.686537 31007 catalog_manager.cc:5495] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6c0f9dc1e994b2d96ab0b3a25f66ef4 (127.21.235.67). New cstate: current_term: 1 leader_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } health_report { overall_health: HEALTHY } } }
I20260809 12:57:12.757755 31628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:12.748790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:41429 (local address 127.0.0.1:35084)
0809 12:57:12.749134 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:12.749157 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:12.749310 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:12.751276 (+ 1966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:12.751281 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:12.751296 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:57:12.751513 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:12.751524 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.753154 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.753161 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:12.754841 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:12.754856 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.755877 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.755884 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:12.756082 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:12.756161 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:57:12.757323 (+ 1162us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:57:12.757348 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 12:57:12.757426 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 12:57:12.757543 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172}
I20260809 12:57:12.758106 31647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:12.748826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35084 (local address 127.21.235.67:41429)
0809 12:57:12.750836 (+ 2010us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:12.750842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:12.750861 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:57:12.750958 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:12.750964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:12.751057 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:12.751227 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:57:12.751683 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.753002 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.755001 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.755700 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.755833 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:12.756416 (+ 583us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:57:12.757199 (+ 783us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:57:12.757359 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:12.757467 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 12:57:12.757549 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1877,"thread_start_us":1741,"threads_started":1}
I20260809 12:57:12.762810 31340 raft_consensus.cc:1261] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f [term 1 FOLLOWER]: Refusing update from remote peer a6c0f9dc1e994b2d96ab0b3a25f66ef4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:57:12.763232 31187 raft_consensus.cc:1261] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f [term 1 FOLLOWER]: Refusing update from remote peer a6c0f9dc1e994b2d96ab0b3a25f66ef4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:57:12.763622 31638 consensus_queue.cc:1035] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:57:12.764099 31597 consensus_queue.cc:1035] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:57:12.769774 31596 tablet_replica.cc:439] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f: TxnStatusTablet state changed. Reason: New leader a6c0f9dc1e994b2d96ab0b3a25f66ef4. Latest consensus state: current_term: 1 leader_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } }
I20260809 12:57:12.770076 31596 tablet_replica.cc:442] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:57:12.781179 31597 tablet_replica.cc:439] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } }
I20260809 12:57:12.781490 31597 tablet_replica.cc:442] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:57:12.785127 31596 tablet_replica.cc:439] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } }
I20260809 12:57:12.785387 31596 tablet_replica.cc:442] T db135aba68aa4207b82c68b3b188f0c1 P bd14937965824ea7a81d2c51b706a77f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:57:12.787542 31598 tablet_replica.cc:439] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f: TxnStatusTablet state changed. Reason: New leader a6c0f9dc1e994b2d96ab0b3a25f66ef4. Latest consensus state: current_term: 1 leader_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } }
I20260809 12:57:12.787850 31598 tablet_replica.cc:442] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:57:12.790814 31638 tablet_replica.cc:439] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } }
I20260809 12:57:12.791563 31638 tablet_replica.cc:442] T db135aba68aa4207b82c68b3b188f0c1 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:57:12.799156 31598 tablet_replica.cc:439] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } } }
I20260809 12:57:12.799429 31598 tablet_replica.cc:442] T db135aba68aa4207b82c68b3b188f0c1 P 1b16ecc6081944b688b42d8afb5a861f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:57:12.814100 31647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:12.808294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:51531 (local address 127.21.235.67:41429)
0809 12:57:12.808452 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:12.808457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:12.808928 (+ 471us) server_negotiation.cc:407] Connection header received
0809 12:57:12.809066 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:12.809072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:12.809158 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:12.809255 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:57:12.809669 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.810864 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.812300 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.812790 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.812926 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:12.813356 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:57:12.813459 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:57:12.813560 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:12.813692 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 12:57:12.813774 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 12:57:12.814209 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:12.808311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:41429 (local address 127.21.235.65:51531)
0809 12:57:12.808821 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:12.808842 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:12.808954 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:12.809273 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:12.809276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:12.809285 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:57:12.809538 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:12.809545 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.810997 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.811001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:12.812143 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:12.812172 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.812948 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.812954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:12.813109 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:12.813162 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:57:12.813562 (+ 400us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:57:12.813576 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 12:57:12.813670 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 12:57:12.813815 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":254,"threads_started":1}
I20260809 12:57:12.823768 31598 consensus_queue.cc:1035] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:57:12.826725 31507 raft_consensus.cc:1261] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 FOLLOWER]: Refusing update from remote peer 1b16ecc6081944b688b42d8afb5a861f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:57:12.830842 31663 consensus_queue.cc:1035] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:57:12.868125 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:12.860281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:35083 (local address 127.21.235.65:42239)
0809 12:57:12.860519 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:12.860526 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:12.861352 (+ 826us) server_negotiation.cc:407] Connection header received
0809 12:57:12.861670 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:12.861676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:12.861764 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:12.861857 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:57:12.862763 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.863997 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.866383 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.866846 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.866956 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:12.867490 (+ 534us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:57:12.867585 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:57:12.867651 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:12.867803 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 12:57:12.867864 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 12:57:12.868597 31667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:12.860291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:42239 (local address 127.21.235.67:35083)
0809 12:57:12.861245 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:12.861269 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:12.861498 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:12.861889 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:12.861894 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:12.861906 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:57:12.862581 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:12.862596 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.864128 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.864132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:12.866249 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:12.866265 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.867048 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.867055 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:12.867251 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:12.867325 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:57:12.867680 (+ 355us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:57:12.867695 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 12:57:12.867779 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 12:57:12.867945 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":535,"threads_started":1}
I20260809 12:57:12.906266 31009 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:57:12.906569 31009 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:57:12.909859 31009 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99: Sending DeleteTablet for 3 replicas of tablet a7210e2b7e5c44a8b28af4f88a644962
I20260809 12:57:12.910723 31166 tablet_service.cc:1514] Processing DeleteTablet for tablet a7210e2b7e5c44a8b28af4f88a644962 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:12 UTC) from {username='oryx', principal='oryx/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:37914
I20260809 12:57:12.910934 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet a7210e2b7e5c44a8b28af4f88a644962 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:12 UTC) from {username='oryx', principal='oryx/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:35076
I20260809 12:57:12.911208 31320 tablet_service.cc:1514] Processing DeleteTablet for tablet a7210e2b7e5c44a8b28af4f88a644962 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:12 UTC) from {username='oryx', principal='oryx/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:60100
I20260809 12:57:12.920796 31671 tablet_replica.cc:330] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: stopping tablet replica
I20260809 12:57:12.921183 31671 raft_consensus.cc:2229] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:57:12.921568 31671 raft_consensus.cc:2258] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:12.923026 31671 ts_tablet_manager.cc:1905] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:12.926803 31672 tablet_replica.cc:330] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f: stopping tablet replica
I20260809 12:57:12.927018 31672 raft_consensus.cc:2229] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:57:12.927278 31672 raft_consensus.cc:2258] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:12.929141 31670 tablet_replica.cc:330] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f: stopping tablet replica
I20260809 12:57:12.929354 31670 raft_consensus.cc:2229] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:57:12.929708 31670 raft_consensus.cc:2258] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:12.930369 31672 ts_tablet_manager.cc:1905] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:12.931113 31670 ts_tablet_manager.cc:1905] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:12.937809 31671 ts_tablet_manager.cc:1918] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:57:12.937975 31671 log.cc:1196] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/a7210e2b7e5c44a8b28af4f88a644962
I20260809 12:57:12.938455 31672 ts_tablet_manager.cc:1918] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:57:12.938495 31671 ts_tablet_manager.cc:1939] T a7210e2b7e5c44a8b28af4f88a644962 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Deleting consensus metadata
I20260809 12:57:12.938688 31672 log.cc:1196] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/a7210e2b7e5c44a8b28af4f88a644962
I20260809 12:57:12.939165 31672 ts_tablet_manager.cc:1939] T a7210e2b7e5c44a8b28af4f88a644962 P bd14937965824ea7a81d2c51b706a77f: Deleting consensus metadata
I20260809 12:57:12.939808 30992 catalog_manager.cc:4841] TS a6c0f9dc1e994b2d96ab0b3a25f66ef4 (127.21.235.67:41429): tablet a7210e2b7e5c44a8b28af4f88a644962 (table test-table [id=bc2d8dea17bb41e18bfec6d3fc8fca25]) successfully deleted
I20260809 12:57:12.941011 30994 catalog_manager.cc:4841] TS bd14937965824ea7a81d2c51b706a77f (127.21.235.66:43139): tablet a7210e2b7e5c44a8b28af4f88a644962 (table test-table [id=bc2d8dea17bb41e18bfec6d3fc8fca25]) successfully deleted
I20260809 12:57:12.943050 31670 ts_tablet_manager.cc:1918] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:57:12.943192 31670 log.cc:1196] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/a7210e2b7e5c44a8b28af4f88a644962
I20260809 12:57:12.943642 31670 ts_tablet_manager.cc:1939] T a7210e2b7e5c44a8b28af4f88a644962 P 1b16ecc6081944b688b42d8afb5a861f: Deleting consensus metadata
I20260809 12:57:12.944697 30992 catalog_manager.cc:4841] TS 1b16ecc6081944b688b42d8afb5a861f (127.21.235.65:42239): tablet a7210e2b7e5c44a8b28af4f88a644962 (table test-table [id=bc2d8dea17bb41e18bfec6d3fc8fca25]) successfully deleted
Aug 09 12:57:12 dist-test-slave-gp6n krb5kdc[30960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 12:57:12 dist-test-slave-gp6n krb5kdc[30960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:12 dist-test-slave-gp6n krb5kdc[30960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 12:57:13.002112 31680 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:12.976774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43146 (local address 127.21.235.126:43003)
0809 12:57:12.986821 (+ 10047us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:12.986828 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:12.987420 (+ 592us) server_negotiation.cc:407] Connection header received
0809 12:57:12.987662 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:12.987668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:12.987764 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:12.987886 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:12.989472 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.990279 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.991959 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:12.992398 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:12.992534 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:13.001776 (+ 9242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43146: BlockingRecv error: recv got EOF from 127.0.0.1:43146 (error 108)
Metrics: {"server-negotiator.queue_time_us":9896,"thread_start_us":10501,"threads_started":1}
Aug 09 12:57:13 dist-test-slave-gp6n krb5kdc[30960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM
I20260809 12:57:13.019891 31680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:13.003564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.21.235.126:43003)
0809 12:57:13.003752 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:13.003758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:13.003996 (+ 238us) server_negotiation.cc:407] Connection header received
0809 12:57:13.004104 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:13.004110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:13.004203 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:13.004342 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:13.005561 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.006856 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.008282 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.008985 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.009137 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:13.013084 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:13.013092 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:13.013098 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:13.013143 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:13.016985 (+ 3842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:13.017636 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:13.017643 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:13.017647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:13.017849 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:13.018226 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:13.018232 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:13.018235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:13.018653 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:13.019020 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:13.019163 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 12:57:13.019421 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:57:13.023608 31009 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:57:13.024228 31009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:57:13.031263 31166 tablet_service.cc:1467] Processing CreateTablet for tablet bdcf2f298e354d1a8f5590e81584a3e0 (DEFAULT_TABLE table=test-table [id=0183bf6903544dccaaa82e05e8ae691f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:13.031540 31475 tablet_service.cc:1467] Processing CreateTablet for tablet bdcf2f298e354d1a8f5590e81584a3e0 (DEFAULT_TABLE table=test-table [id=0183bf6903544dccaaa82e05e8ae691f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:13.031757 31166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdcf2f298e354d1a8f5590e81584a3e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:13.032045 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdcf2f298e354d1a8f5590e81584a3e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:13.036638 31688 tablet_bootstrap.cc:492] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f: Bootstrap starting.
I20260809 12:57:13.036653 31320 tablet_service.cc:1467] Processing CreateTablet for tablet bdcf2f298e354d1a8f5590e81584a3e0 (DEFAULT_TABLE table=test-table [id=0183bf6903544dccaaa82e05e8ae691f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:13.037096 31320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdcf2f298e354d1a8f5590e81584a3e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:13.038820 31688 tablet_bootstrap.cc:654] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:13.047261 31691 tablet_bootstrap.cc:492] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f: Bootstrap starting.
I20260809 12:57:13.049394 31688 tablet_bootstrap.cc:492] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f: No bootstrap required, opened a new log
I20260809 12:57:13.049664 31689 tablet_bootstrap.cc:492] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Bootstrap starting.
I20260809 12:57:13.049664 31688 ts_tablet_manager.cc:1397] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.004s
I20260809 12:57:13.050631 31688 raft_consensus.cc:348] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:13.051046 31688 raft_consensus.cc:374] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:13.051196 31688 raft_consensus.cc:729] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b16ecc6081944b688b42d8afb5a861f, State: Initialized, Role: FOLLOWER
I20260809 12:57:13.051537 31688 consensus_queue.cc:260] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:13.051797 31691 tablet_bootstrap.cc:654] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:13.051996 31689 tablet_bootstrap.cc:654] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:13.056360 31688 ts_tablet_manager.cc:1428] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260809 12:57:13.061512 31689 tablet_bootstrap.cc:492] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: No bootstrap required, opened a new log
I20260809 12:57:13.061693 31689 ts_tablet_manager.cc:1397] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260809 12:57:13.061859 31691 tablet_bootstrap.cc:492] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f: No bootstrap required, opened a new log
I20260809 12:57:13.062047 31691 ts_tablet_manager.cc:1397] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.009s
I20260809 12:57:13.062598 31689 raft_consensus.cc:348] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:13.062811 31689 raft_consensus.cc:374] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:13.062968 31689 raft_consensus.cc:729] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6c0f9dc1e994b2d96ab0b3a25f66ef4, State: Initialized, Role: FOLLOWER
I20260809 12:57:13.062973 31691 raft_consensus.cc:348] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:13.063264 31691 raft_consensus.cc:374] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:13.063376 31691 raft_consensus.cc:729] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd14937965824ea7a81d2c51b706a77f, State: Initialized, Role: FOLLOWER
I20260809 12:57:13.063272 31689 consensus_queue.cc:260] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:13.063627 31691 consensus_queue.cc:260] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:13.064100 31689 ts_tablet_manager.cc:1428] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 12:57:13.064281 31691 ts_tablet_manager.cc:1428] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 12:57:13.089099 31596 raft_consensus.cc:482] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:57:13.089382 31596 raft_consensus.cc:504] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:13.091125 31596 leader_election.cc:288] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b16ecc6081944b688b42d8afb5a861f (127.21.235.65:42239), a6c0f9dc1e994b2d96ab0b3a25f66ef4 (127.21.235.67:41429)
I20260809 12:57:13.098897 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:13.091608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:44313 (local address 127.21.235.65:42239)
0809 12:57:13.091813 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:13.091820 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:13.092042 (+ 222us) server_negotiation.cc:407] Connection header received
0809 12:57:13.092372 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:13.092382 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:13.092523 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:13.092601 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:13.093474 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.094480 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.096785 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.097713 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.097910 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:13.098002 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:13.098450 (+ 448us) server_negotiation.cc:299] Negotiation successful
0809 12:57:13.098586 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 12:57:13.098968 31694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:13.091360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:42239 (local address 127.21.235.66:44313)
0809 12:57:13.091938 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:13.091960 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:13.092153 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:13.092825 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:13.092830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:13.092861 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:13.093249 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:13.093265 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.094601 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.094611 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:13.096629 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:13.096641 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.098017 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.098028 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:13.098155 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:13.098186 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 12:57:13.098387 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 12:57:13.098552 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":305,"threads_started":1}
I20260809 12:57:13.099709 31187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdcf2f298e354d1a8f5590e81584a3e0" candidate_uuid: "bd14937965824ea7a81d2c51b706a77f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b16ecc6081944b688b42d8afb5a861f" is_pre_election: true
I20260809 12:57:13.099995 31187 raft_consensus.cc:2454] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd14937965824ea7a81d2c51b706a77f in term 0.
I20260809 12:57:13.100500 31267 leader_election.cc:302] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b16ecc6081944b688b42d8afb5a861f, bd14937965824ea7a81d2c51b706a77f; no voters:
I20260809 12:57:13.100956 31596 raft_consensus.cc:2781] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:57:13.101086 31596 raft_consensus.cc:482] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:57:13.101176 31596 raft_consensus.cc:3037] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:13.101377 31647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:13.092314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:34211 (local address 127.21.235.67:41429)
0809 12:57:13.092498 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:13.092504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:13.093405 (+ 901us) server_negotiation.cc:407] Connection header received
0809 12:57:13.093718 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:13.093724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:13.093803 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:13.093888 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:13.094876 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.096096 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.098630 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.100031 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.100183 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:13.100262 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:13.100915 (+ 653us) server_negotiation.cc:299] Negotiation successful
0809 12:57:13.101065 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 12:57:13.101990 31695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:13.092106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:41429 (local address 127.21.235.66:34211)
0809 12:57:13.093289 (+ 1183us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:13.093311 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:13.093521 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:13.094186 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:13.094192 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:13.094223 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:13.094650 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:13.094663 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.096269 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.096275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:13.098427 (+ 2152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:13.098444 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.100505 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.100511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:13.100704 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:13.100737 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 12:57:13.101579 (+ 842us) client_negotiation.cc:240] Negotiation successful
0809 12:57:13.101706 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"thread_start_us":549,"threads_started":1}
I20260809 12:57:13.102861 31507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdcf2f298e354d1a8f5590e81584a3e0" candidate_uuid: "bd14937965824ea7a81d2c51b706a77f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" is_pre_election: true
I20260809 12:57:13.103152 31507 raft_consensus.cc:2454] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd14937965824ea7a81d2c51b706a77f in term 0.
I20260809 12:57:13.104730 31596 raft_consensus.cc:504] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:13.105505 31596 leader_election.cc:288] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [CANDIDATE]: Term 1 election: Requested vote from peers 1b16ecc6081944b688b42d8afb5a861f (127.21.235.65:42239), a6c0f9dc1e994b2d96ab0b3a25f66ef4 (127.21.235.67:41429)
I20260809 12:57:13.105976 31187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdcf2f298e354d1a8f5590e81584a3e0" candidate_uuid: "bd14937965824ea7a81d2c51b706a77f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b16ecc6081944b688b42d8afb5a861f"
I20260809 12:57:13.106045 31507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdcf2f298e354d1a8f5590e81584a3e0" candidate_uuid: "bd14937965824ea7a81d2c51b706a77f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4"
I20260809 12:57:13.106233 31187 raft_consensus.cc:3037] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:13.106288 31507 raft_consensus.cc:3037] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:13.108587 31187 raft_consensus.cc:2454] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd14937965824ea7a81d2c51b706a77f in term 1.
I20260809 12:57:13.109048 31267 leader_election.cc:302] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b16ecc6081944b688b42d8afb5a861f, bd14937965824ea7a81d2c51b706a77f; no voters:
I20260809 12:57:13.109583 31507 raft_consensus.cc:2454] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd14937965824ea7a81d2c51b706a77f in term 1.
I20260809 12:57:13.109942 31596 raft_consensus.cc:2781] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:57:13.110549 31596 raft_consensus.cc:686] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [term 1 LEADER]: Becoming Leader. State: Replica: bd14937965824ea7a81d2c51b706a77f, State: Running, Role: LEADER
I20260809 12:57:13.111047 31596 consensus_queue.cc:237] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:13.115401 31008 catalog_manager.cc:5495] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f reported cstate change: term changed from 0 to 1, leader changed from <none> to bd14937965824ea7a81d2c51b706a77f (127.21.235.66). New cstate: current_term: 1 leader_uuid: "bd14937965824ea7a81d2c51b706a77f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } health_report { overall_health: UNKNOWN } } }
I20260809 12:57:13.136866 31635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:13.131238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60110 (local address 127.21.235.66:43139)
0809 12:57:13.131436 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:13.131445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:13.131672 (+ 227us) server_negotiation.cc:407] Connection header received
0809 12:57:13.131934 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:13.131938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:13.132029 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:13.132187 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:57:13.132568 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.133283 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.134687 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.135087 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.135222 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:13.135685 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:57:13.136276 (+ 591us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:57:13.136436 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:13.136517 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 12:57:13.136601 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 12:57:13.141111 31507 raft_consensus.cc:1261] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 FOLLOWER]: Refusing update from remote peer bd14937965824ea7a81d2c51b706a77f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:57:13.141176 31187 raft_consensus.cc:1261] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f [term 1 FOLLOWER]: Refusing update from remote peer bd14937965824ea7a81d2c51b706a77f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:57:13.141824 31697 consensus_queue.cc:1035] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:57:13.142541 31596 consensus_queue.cc:1035] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:57:13.162693 31008 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:57:13.162977 31008 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:57:13.168061 31008 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99: Sending DeleteTablet for 3 replicas of tablet bdcf2f298e354d1a8f5590e81584a3e0
I20260809 12:57:13.169821 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet bdcf2f298e354d1a8f5590e81584a3e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:13 UTC) from {username='oryx', principal='oryx/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:35076
I20260809 12:57:13.170153 31671 tablet_replica.cc:330] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: stopping tablet replica
I20260809 12:57:13.170361 31671 raft_consensus.cc:2229] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:57:13.170643 31671 raft_consensus.cc:2258] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:13.171208 31166 tablet_service.cc:1514] Processing DeleteTablet for tablet bdcf2f298e354d1a8f5590e81584a3e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:13 UTC) from {username='oryx', principal='oryx/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:37914
I20260809 12:57:13.171515 31670 tablet_replica.cc:330] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f: stopping tablet replica
I20260809 12:57:13.171353 31009 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 12:57:13.171715 31670 raft_consensus.cc:2229] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:57:13.171873 31670 pending_rounds.cc:62] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f: Trying to abort 2 pending ops.
W20260809 12:57:13.172039 31009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:57:13.172057 31670 pending_rounds.cc:68] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316603834938589184 op_type: WRITE_OP write_request { tablet_id: "bdcf2f298e354d1a8f5590e81584a3e0" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\325\364\341\323\006\"8\n\ntest-admin\022*\n 0183bf6903544dccaaa82e05e8ae691f\020\001\030\001 \001(\001" signature: "<redacted>""\025#!6\251ah\027\350\340\\\337\360\036\023\017\357R\"\301\323[u\253\237/\315\330$]\234\014\357\304@8\2733\034{Y\024D\365\231\310J\033\370\354`_\244\250\376\035zp\234\356]\007rF" signing_key_seq_num: 0 } } request_id { client_id: "dc78ea2d06e64d5881f1fcbf125cdf16" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260809 12:57:13.172950 31670 pending_rounds.cc:68] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316603834828541952 op_type: NO_OP noop_request { }
I20260809 12:57:13.173110 31670 raft_consensus.cc:2866] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260809 12:57:13.173233 31670 raft_consensus.cc:2258] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:13.174501 31670 ts_tablet_manager.cc:1905] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:13.175616 31320 tablet_service.cc:1514] Processing DeleteTablet for tablet bdcf2f298e354d1a8f5590e81584a3e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:13 UTC) from {username='oryx', principal='oryx/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:60100
I20260809 12:57:13.175973 31672 tablet_replica.cc:330] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f: stopping tablet replica
I20260809 12:57:13.176203 31672 raft_consensus.cc:2229] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:57:13.176677 31672 raft_consensus.cc:2258] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:13.181674 31670 ts_tablet_manager.cc:1918] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:57:13.181844 31670 log.cc:1196] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/bdcf2f298e354d1a8f5590e81584a3e0
I20260809 12:57:13.182291 31670 ts_tablet_manager.cc:1939] T bdcf2f298e354d1a8f5590e81584a3e0 P 1b16ecc6081944b688b42d8afb5a861f: Deleting consensus metadata
I20260809 12:57:13.183533 31166 tablet_service.cc:1467] Processing CreateTablet for tablet 9e621412cfe140469725c10b2fa1cbbd (DEFAULT_TABLE table=test-table [id=dec88f8bb2694db78b3173b9623d5304]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:13.183624 31672 ts_tablet_manager.cc:1905] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:13.183635 31475 tablet_service.cc:1467] Processing CreateTablet for tablet 9e621412cfe140469725c10b2fa1cbbd (DEFAULT_TABLE table=test-table [id=dec88f8bb2694db78b3173b9623d5304]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:13.183969 31166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e621412cfe140469725c10b2fa1cbbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:13.184029 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e621412cfe140469725c10b2fa1cbbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:13.190267 31688 tablet_bootstrap.cc:492] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f: Bootstrap starting.
I20260809 12:57:13.191191 31320 tablet_service.cc:1467] Processing CreateTablet for tablet 9e621412cfe140469725c10b2fa1cbbd (DEFAULT_TABLE table=test-table [id=dec88f8bb2694db78b3173b9623d5304]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:13.191668 31320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e621412cfe140469725c10b2fa1cbbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:13.191879 31688 tablet_bootstrap.cc:654] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:13.199118 31689 tablet_bootstrap.cc:492] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Bootstrap starting.
I20260809 12:57:13.199766 30992 catalog_manager.cc:4841] TS 1b16ecc6081944b688b42d8afb5a861f (127.21.235.65:42239): tablet bdcf2f298e354d1a8f5590e81584a3e0 (table test-table [id=0183bf6903544dccaaa82e05e8ae691f]) successfully deleted
I20260809 12:57:13.200847 31691 tablet_bootstrap.cc:492] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f: Bootstrap starting.
I20260809 12:57:13.201048 31689 tablet_bootstrap.cc:654] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:13.203153 31691 tablet_bootstrap.cc:654] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:13.203433 31688 tablet_bootstrap.cc:492] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f: No bootstrap required, opened a new log
I20260809 12:57:13.203614 31688 ts_tablet_manager.cc:1397] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.004s
I20260809 12:57:13.204543 31688 raft_consensus.cc:348] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:13.204974 31688 raft_consensus.cc:374] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:13.205084 31688 raft_consensus.cc:729] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b16ecc6081944b688b42d8afb5a861f, State: Initialized, Role: FOLLOWER
I20260809 12:57:13.205422 31688 consensus_queue.cc:260] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:13.208763 31688 ts_tablet_manager.cc:1428] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260809 12:57:13.208969 31671 ts_tablet_manager.cc:1905] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:13.218163 31671 ts_tablet_manager.cc:1918] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:57:13.219287 31671 log.cc:1196] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/bdcf2f298e354d1a8f5590e81584a3e0
I20260809 12:57:13.219609 31689 tablet_bootstrap.cc:492] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4: No bootstrap required, opened a new log
I20260809 12:57:13.219802 31689 ts_tablet_manager.cc:1397] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.000s
I20260809 12:57:13.220664 31691 tablet_bootstrap.cc:492] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f: No bootstrap required, opened a new log
I20260809 12:57:13.220759 31689 raft_consensus.cc:348] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:13.221071 31689 raft_consensus.cc:374] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:13.221171 31689 raft_consensus.cc:729] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6c0f9dc1e994b2d96ab0b3a25f66ef4, State: Initialized, Role: FOLLOWER
I20260809 12:57:13.221447 31691 ts_tablet_manager.cc:1397] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.013s
I20260809 12:57:13.221464 31689 consensus_queue.cc:260] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:13.221940 31671 ts_tablet_manager.cc:1939] T bdcf2f298e354d1a8f5590e81584a3e0 P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Deleting consensus metadata
I20260809 12:57:13.222154 31689 ts_tablet_manager.cc:1428] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 12:57:13.222283 31691 raft_consensus.cc:348] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:13.222582 31691 raft_consensus.cc:374] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:13.222687 31691 raft_consensus.cc:729] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd14937965824ea7a81d2c51b706a77f, State: Initialized, Role: FOLLOWER
I20260809 12:57:13.226612 31691 consensus_queue.cc:260] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:13.227276 31691 ts_tablet_manager.cc:1428] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260809 12:57:13.227835 31672 ts_tablet_manager.cc:1918] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:57:13.227986 31672 log.cc:1196] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/bdcf2f298e354d1a8f5590e81584a3e0
I20260809 12:57:13.229763 31672 ts_tablet_manager.cc:1939] T bdcf2f298e354d1a8f5590e81584a3e0 P bd14937965824ea7a81d2c51b706a77f: Deleting consensus metadata
I20260809 12:57:13.229915 30992 catalog_manager.cc:4841] TS a6c0f9dc1e994b2d96ab0b3a25f66ef4 (127.21.235.67:41429): tablet bdcf2f298e354d1a8f5590e81584a3e0 (table test-table [id=0183bf6903544dccaaa82e05e8ae691f]) successfully deleted
I20260809 12:57:13.230942 30994 catalog_manager.cc:4841] TS bd14937965824ea7a81d2c51b706a77f (127.21.235.66:43139): tablet bdcf2f298e354d1a8f5590e81584a3e0 (table test-table [id=0183bf6903544dccaaa82e05e8ae691f]) successfully deleted
I20260809 12:57:13.409647 31669 raft_consensus.cc:482] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:57:13.409905 31669 raft_consensus.cc:504] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:13.410754 31669 leader_election.cc:288] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd14937965824ea7a81d2c51b706a77f (127.21.235.66:43139), 1b16ecc6081944b688b42d8afb5a861f (127.21.235.65:42239)
I20260809 12:57:13.411535 31340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e621412cfe140469725c10b2fa1cbbd" candidate_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd14937965824ea7a81d2c51b706a77f" is_pre_election: true
I20260809 12:57:13.411521 31187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e621412cfe140469725c10b2fa1cbbd" candidate_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b16ecc6081944b688b42d8afb5a861f" is_pre_election: true
I20260809 12:57:13.411873 31340 raft_consensus.cc:2454] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6c0f9dc1e994b2d96ab0b3a25f66ef4 in term 0.
I20260809 12:57:13.411868 31187 raft_consensus.cc:2454] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6c0f9dc1e994b2d96ab0b3a25f66ef4 in term 0.
I20260809 12:57:13.412408 31423 leader_election.cc:302] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b16ecc6081944b688b42d8afb5a861f, a6c0f9dc1e994b2d96ab0b3a25f66ef4; no voters:
I20260809 12:57:13.412747 31669 raft_consensus.cc:2781] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:57:13.412894 31669 raft_consensus.cc:482] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:57:13.413000 31669 raft_consensus.cc:3037] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:13.415534 31669 raft_consensus.cc:504] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:13.416142 31669 leader_election.cc:288] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [CANDIDATE]: Term 1 election: Requested vote from peers bd14937965824ea7a81d2c51b706a77f (127.21.235.66:43139), 1b16ecc6081944b688b42d8afb5a861f (127.21.235.65:42239)
I20260809 12:57:13.416764 31187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e621412cfe140469725c10b2fa1cbbd" candidate_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b16ecc6081944b688b42d8afb5a861f"
I20260809 12:57:13.416770 31340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e621412cfe140469725c10b2fa1cbbd" candidate_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd14937965824ea7a81d2c51b706a77f"
I20260809 12:57:13.416980 31187 raft_consensus.cc:3037] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:13.417037 31340 raft_consensus.cc:3037] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:13.419159 31187 raft_consensus.cc:2454] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6c0f9dc1e994b2d96ab0b3a25f66ef4 in term 1.
I20260809 12:57:13.419625 31423 leader_election.cc:302] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b16ecc6081944b688b42d8afb5a861f, a6c0f9dc1e994b2d96ab0b3a25f66ef4; no voters:
I20260809 12:57:13.419961 31669 raft_consensus.cc:2781] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:57:13.420533 31669 raft_consensus.cc:686] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 LEADER]: Becoming Leader. State: Replica: a6c0f9dc1e994b2d96ab0b3a25f66ef4, State: Running, Role: LEADER
I20260809 12:57:13.420863 31340 raft_consensus.cc:2454] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6c0f9dc1e994b2d96ab0b3a25f66ef4 in term 1.
I20260809 12:57:13.420944 31669 consensus_queue.cc:237] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } }
I20260809 12:57:13.424010 31007 catalog_manager.cc:5495] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6c0f9dc1e994b2d96ab0b3a25f66ef4 (127.21.235.67). New cstate: current_term: 1 leader_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6c0f9dc1e994b2d96ab0b3a25f66ef4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41429 } health_report { overall_health: HEALTHY } } }
I20260809 12:57:13.454109 31647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:13.448371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35090 (local address 127.21.235.67:41429)
0809 12:57:13.448560 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:13.448566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:13.448782 (+ 216us) server_negotiation.cc:407] Connection header received
0809 12:57:13.449000 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:13.449004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:13.449066 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:13.449137 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:57:13.449599 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.450281 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.452314 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.452749 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.452848 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:13.453373 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:57:13.453483 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:57:13.453549 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:13.453722 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 12:57:13.453787 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 12:57:13.467947 31647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:13.462333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:33393 (local address 127.21.235.67:41429)
0809 12:57:13.462566 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:13.462575 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:13.463097 (+ 522us) server_negotiation.cc:407] Connection header received
0809 12:57:13.463321 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:13.463326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:13.463442 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:13.463530 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:57:13.463905 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.464772 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.465990 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.466446 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.466580 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:13.467077 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:57:13.467191 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:57:13.467273 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:13.467440 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 12:57:13.467544 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 12:57:13.467978 31715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:13.462262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:41429 (local address 127.21.235.67:33393)
0809 12:57:13.462997 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:13.463012 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:13.463201 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:13.463540 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:13.463544 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:13.463554 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:57:13.463782 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:13.463791 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.464898 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.464905 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:13.465845 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:13.465856 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.466614 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.466640 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:13.466841 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:13.466892 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:57:13.467293 (+ 401us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:57:13.467305 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 12:57:13.467396 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 12:57:13.467556 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":3,"thread_start_us":471,"threads_started":1}
I20260809 12:57:13.476454 31340 raft_consensus.cc:1261] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f [term 1 FOLLOWER]: Refusing update from remote peer a6c0f9dc1e994b2d96ab0b3a25f66ef4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:57:13.476944 31669 consensus_queue.cc:1035] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd14937965824ea7a81d2c51b706a77f" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 43139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:57:13.491918 31187 raft_consensus.cc:1261] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f [term 1 FOLLOWER]: Refusing update from remote peer a6c0f9dc1e994b2d96ab0b3a25f66ef4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:57:13.494323 31710 consensus_queue.cc:1035] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b16ecc6081944b688b42d8afb5a861f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:57:13.518991 31650 mvcc.cc:205] Tried to move back new op lower bound from 7316603836351582208 to 7316603836094906368. Current Snapshot: MvccSnapshot[applied={T|T < 7316603836351582208}]
I20260809 12:57:13.562090 31007 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:57:13.562379 31007 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:57:13.566435 31007 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ba10c24ebcc427481e7e33b53805c99: Sending DeleteTablet for 3 replicas of tablet 9e621412cfe140469725c10b2fa1cbbd
I20260809 12:57:13.567328 31320 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e621412cfe140469725c10b2fa1cbbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:13 UTC) from {username='oryx', principal='oryx/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:60100
I20260809 12:57:13.567598 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e621412cfe140469725c10b2fa1cbbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:13 UTC) from {username='oryx', principal='oryx/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:35076
I20260809 12:57:13.567659 31672 tablet_replica.cc:330] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f: stopping tablet replica
I20260809 12:57:13.567852 31672 raft_consensus.cc:2229] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:57:13.567896 31671 tablet_replica.cc:330] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4: stopping tablet replica
I20260809 12:57:13.568076 31672 raft_consensus.cc:2258] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:13.568089 31671 raft_consensus.cc:2229] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:57:13.568614 31671 raft_consensus.cc:2258] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:13.568769 31166 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e621412cfe140469725c10b2fa1cbbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:13 UTC) from {username='oryx', principal='oryx/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:37914
I20260809 12:57:13.569098 31670 tablet_replica.cc:330] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f: stopping tablet replica
I20260809 12:57:13.569298 31670 raft_consensus.cc:2229] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:57:13.569520 31670 raft_consensus.cc:2258] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:13.569742 31671 ts_tablet_manager.cc:1905] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:13.570604 31670 ts_tablet_manager.cc:1905] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:13.577090 31671 ts_tablet_manager.cc:1918] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:57:13.577224 31671 log.cc:1196] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/9e621412cfe140469725c10b2fa1cbbd
I20260809 12:57:13.577358 31670 ts_tablet_manager.cc:1918] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:57:13.577489 31670 log.cc:1196] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/9e621412cfe140469725c10b2fa1cbbd
I20260809 12:57:13.577663 31671 ts_tablet_manager.cc:1939] T 9e621412cfe140469725c10b2fa1cbbd P a6c0f9dc1e994b2d96ab0b3a25f66ef4: Deleting consensus metadata
I20260809 12:57:13.577919 31670 ts_tablet_manager.cc:1939] T 9e621412cfe140469725c10b2fa1cbbd P 1b16ecc6081944b688b42d8afb5a861f: Deleting consensus metadata
I20260809 12:57:13.579001 30992 catalog_manager.cc:4841] TS a6c0f9dc1e994b2d96ab0b3a25f66ef4 (127.21.235.67:41429): tablet 9e621412cfe140469725c10b2fa1cbbd (table test-table [id=dec88f8bb2694db78b3173b9623d5304]) successfully deleted
I20260809 12:57:13.580093 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 31088
I20260809 12:57:13.584252 31672 ts_tablet_manager.cc:1905] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:13.591958 31672 ts_tablet_manager.cc:1918] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:57:13.592161 31672 log.cc:1196] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/9e621412cfe140469725c10b2fa1cbbd
I20260809 12:57:13.592670 31672 ts_tablet_manager.cc:1939] T 9e621412cfe140469725c10b2fa1cbbd P bd14937965824ea7a81d2c51b706a77f: Deleting consensus metadata
I20260809 12:57:13.593508 30992 catalog_manager.cc:4841] TS 1b16ecc6081944b688b42d8afb5a861f (127.21.235.65:42239): tablet 9e621412cfe140469725c10b2fa1cbbd (table test-table [id=dec88f8bb2694db78b3173b9623d5304]) successfully deleted
I20260809 12:57:13.594966 30994 catalog_manager.cc:4841] TS bd14937965824ea7a81d2c51b706a77f (127.21.235.66:43139): tablet 9e621412cfe140469725c10b2fa1cbbd (table test-table [id=dec88f8bb2694db78b3173b9623d5304]) successfully deleted
Aug 09 12:57:13 dist-test-slave-gp6n krb5kdc[30960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.235.65@KRBTEST.COM
I20260809 12:57:13.615653 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:13.581869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37918 (local address 127.21.235.65:42239)
0809 12:57:13.582104 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:13.582111 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:13.595859 (+ 13748us) server_negotiation.cc:407] Connection header received
0809 12:57:13.596087 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:13.596094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:13.596181 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:13.596285 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:13.598032 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.599211 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.601359 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.601992 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.602132 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:13.607760 (+ 5628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:13.607789 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:13.607810 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:13.607871 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:13.612204 (+ 4333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:13.613197 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:13.613206 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:13.613211 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:13.613381 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:13.613825 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:13.613832 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:13.613835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:13.614307 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:13.614783 (+ 476us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:13.615036 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 12:57:13.615291 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 12:57:13.616470 31238 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:13.814182 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 31088
I20260809 12:57:13.849881 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 31248
Aug 09 12:57:13 dist-test-slave-gp6n krb5kdc[30960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.235.66@KRBTEST.COM
I20260809 12:57:13.870797 31726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:13.850705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60120 (local address 127.21.235.66:43139)
0809 12:57:13.851716 (+ 1011us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:13.851721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:13.851737 (+ 16us) server_negotiation.cc:407] Connection header received
0809 12:57:13.851792 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:13.851797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:13.851861 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:13.851950 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:13.853820 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.854659 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.856601 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:13.857404 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:13.857571 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:13.863116 (+ 5545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:13.863141 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:13.863163 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:13.863214 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:13.867460 (+ 4246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:13.868243 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:13.868254 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:13.868259 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:13.868436 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:13.868850 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:13.868856 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:13.868860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:13.869352 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:13.869691 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:13.869969 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 12:57:13.870234 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":890,"thread_start_us":765,"threads_started":1}
I20260809 12:57:13.871595 31393 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:14.008718 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 31248
I20260809 12:57:14.035728 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 31402
Aug 09 12:57:14 dist-test-slave-gp6n krb5kdc[30960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.235.67@KRBTEST.COM
I20260809 12:57:14.051798 31728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:14.036693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35098 (local address 127.21.235.67:41429)
0809 12:57:14.037168 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:14.037172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:14.037189 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:57:14.037244 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:14.037247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:14.037308 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:14.037441 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:14.038903 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:14.039645 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:14.040903 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:14.041309 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:14.041404 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:14.045864 (+ 4460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:14.045879 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:14.045896 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:14.045948 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:14.048889 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:14.049597 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:14.049605 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:14.049607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:14.049760 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:14.050137 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:14.050141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:14.050144 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:14.050673 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:14.050951 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:14.051176 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 12:57:14.051437 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":249,"threads_started":1}
I20260809 12:57:14.052438 31556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:14.198498 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 31402
I20260809 12:57:14.228502 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 30974
I20260809 12:57:14.229723 31054 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:14.349653 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 30974
2026-08-09T12:57:14Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6318 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:57:14 dist-test-slave-gp6n krb5kdc[31737](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:57:14 dist-test-slave-gp6n krb5kdc[31737](info): set up 2 sockets
Aug 09 12:57:14 dist-test-slave-gp6n krb5kdc[31737](info): commencing operation
krb5kdc: starting...
W20260809 12:57:15.529584 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:57:15 dist-test-slave-gp6n krb5kdc[31737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:57:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:57:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 12:57:15.781610 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:33031
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:38659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:33031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:57:16.121444 31752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:16.121727 31752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:16.121889 31752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:16.129210 31752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:57:16.129350 31752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:16.129438 31752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:16.129531 31752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:57:16.129611 31752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:57:16.139477 31752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:38659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:33031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:33031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:16.140713 31752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:16.141995 31752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:16.153676 31757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:16.153729 31758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:16.154435 31760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:16.155350 31752 server_base.cc:1034] running on GCE node
I20260809 12:57:16.156092 31752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:16.157500 31752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:16.158725 31752 hybrid_clock.cc:648] HybridClock initialized: now 1786280236158681 us; error 49 us; skew 500 ppm
I20260809 12:57:16.159076 31752 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T12:57:16Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1878 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:57:16 dist-test-slave-gp6n krb5kdc[31768](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:57:16 dist-test-slave-gp6n krb5kdc[31768](info): set up 2 sockets
Aug 09 12:57:16 dist-test-slave-gp6n krb5kdc[31768](info): commencing operation
krb5kdc: starting...
W20260809 12:57:17.387779 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.001s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:57:17 dist-test-slave-gp6n krb5kdc[31768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:57:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:57:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 12:57:17.647046 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:38679
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:38679
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:57:17.985630 31783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:17.985919 31783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:17.986094 31783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:17.993465 31783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:57:17.993578 31783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:17.993682 31783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:17.993782 31783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:57:17.993875 31783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:57:18.004469 31783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:38679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:38679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:18.005847 31783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:18.007637 31783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:18.020524 31788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:18.020471 31789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:18.020552 31792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:18.021421 31783 server_base.cc:1034] running on GCE node
I20260809 12:57:18.022203 31783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:18.023655 31783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:18.024873 31783 hybrid_clock.cc:648] HybridClock initialized: now 1786280238024829 us; error 33 us; skew 500 ppm
I20260809 12:57:18.025245 31783 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:18 dist-test-slave-gp6n krb5kdc[31768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280238, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:18.034003 31783 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:57:18.036032 31783 webserver.cc:460] Webserver started at http://127.21.235.126:34901/ using document root <none> and password file <none>
I20260809 12:57:18.036755 31783 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:18.036890 31783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:18.037233 31783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:18.040616 31783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "05576becfe7f4c8c9dcb71c8f26844f5"
format_stamp: "Formatted at 2026-08-09 12:57:18 on dist-test-slave-gp6n"
server_key: "42fea83420b21474febd49cbaec236ed"
server_key_iv: "ed3c0555ed81a826d05447b9ca3487b3"
server_key_version: "encryptionkey@0"
I20260809 12:57:18.041443 31783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "05576becfe7f4c8c9dcb71c8f26844f5"
format_stamp: "Formatted at 2026-08-09 12:57:18 on dist-test-slave-gp6n"
server_key: "42fea83420b21474febd49cbaec236ed"
server_key_iv: "ed3c0555ed81a826d05447b9ca3487b3"
server_key_version: "encryptionkey@0"
I20260809 12:57:18.047116 31783 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 12:57:18.050623 31801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:18.051957 31783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:57:18.052157 31783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "05576becfe7f4c8c9dcb71c8f26844f5"
format_stamp: "Formatted at 2026-08-09 12:57:18 on dist-test-slave-gp6n"
server_key: "42fea83420b21474febd49cbaec236ed"
server_key_iv: "ed3c0555ed81a826d05447b9ca3487b3"
server_key_version: "encryptionkey@0"
I20260809 12:57:18.052348 31783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:18.075939 31783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:18.080662 31783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:18.080976 31783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:18.100411 31783 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:38679
I20260809 12:57:18.100399 31857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:38679 every 8 connection(s)
I20260809 12:57:18.102195 31783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:57:18.106249 31858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:18.110937 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 31783
I20260809 12:57:18.111166 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:57:18.111886 22445 external_mini_cluster.cc:1428] Setting key 68d4821e0a983e5ed49763e184e81cc7
I20260809 12:57:18.117182 31858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5: Bootstrap starting.
I20260809 12:57:18.120945 31858 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:18.122181 31858 log.cc:824] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5: Log is configured to *not* fsync() on all Append() calls
I20260809 12:57:18.126101 31858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5: No bootstrap required, opened a new log
Aug 09 12:57:18 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:57:18.135089 31858 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05576becfe7f4c8c9dcb71c8f26844f5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 38679 } }
I20260809 12:57:18.135524 31858 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:18.135689 31858 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05576becfe7f4c8c9dcb71c8f26844f5, State: Initialized, Role: FOLLOWER
I20260809 12:57:18.136386 31858 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05576becfe7f4c8c9dcb71c8f26844f5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 38679 } }
I20260809 12:57:18.136659 31858 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:57:18.136842 31858 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:57:18.137037 31858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:18.137131 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:18.113842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56918 (local address 127.21.235.126:38679)
0809 12:57:18.114746 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:18.114760 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:18.114801 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:57:18.116506 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:18.116528 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:18.116912 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:18.117224 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:18.119114 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:18.120467 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:18.122392 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:18.123597 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:18.123770 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:18.129161 (+ 5391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:18.129172 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:18.129190 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:18.129229 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:18.132538 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:18.133292 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:18.133308 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:18.133314 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:18.133505 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:18.133905 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:18.133912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:18.133923 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:18.134562 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:18.134869 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:18.135356 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 12:57:18.135804 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":454,"threads_started":1}
I20260809 12:57:18.139752 31858 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05576becfe7f4c8c9dcb71c8f26844f5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 38679 } }
I20260809 12:57:18.140378 31858 leader_election.cc:302] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05576becfe7f4c8c9dcb71c8f26844f5; no voters:
I20260809 12:57:18.140971 31858 leader_election.cc:288] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:57:18.141155 31863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:57:18.141942 31863 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [term 1 LEADER]: Becoming Leader. State: Replica: 05576becfe7f4c8c9dcb71c8f26844f5, State: Running, Role: LEADER
I20260809 12:57:18.142589 31863 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05576becfe7f4c8c9dcb71c8f26844f5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 38679 } }
I20260809 12:57:18.143278 31858 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:57:18.145378 31864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05576becfe7f4c8c9dcb71c8f26844f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05576becfe7f4c8c9dcb71c8f26844f5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 38679 } } }
I20260809 12:57:18.145787 31864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:18.150573 31865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05576becfe7f4c8c9dcb71c8f26844f5. Latest consensus state: current_term: 1 leader_uuid: "05576becfe7f4c8c9dcb71c8f26844f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05576becfe7f4c8c9dcb71c8f26844f5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 38679 } } }
I20260809 12:57:18.150924 31865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:18.153370 31871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:57:18.159590 31871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:57:18.171018 31871 catalog_manager.cc:1347] Generated new cluster ID: e31d88ce861443e5bcb9c0beb33530ab
I20260809 12:57:18.171144 31871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:57:18.189093 31871 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:57:18.190466 31871 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:57:18.201821 31871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5: Generated new TSK 0
I20260809 12:57:18.202692 31871 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
I20260809 12:57:18.275327 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:38679
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:57:18.614252 31888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:18.614552 31888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:18.614711 31888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:18.622023 31888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:18.622125 31888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:18.622300 31888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:57:18.631963 31888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:38679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:18.633217 31888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:18.634568 31888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:18.647953 31893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:18.647866 31897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:18.647831 31894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:18.648675 31888 server_base.cc:1034] running on GCE node
I20260809 12:57:18.649331 31888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:18.650274 31888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:18.651500 31888 hybrid_clock.cc:648] HybridClock initialized: now 1786280238651473 us; error 77 us; skew 500 ppm
I20260809 12:57:18.651824 31888 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:18 dist-test-slave-gp6n krb5kdc[31768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280238, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:18.655426 31888 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 12:57:18.656994 31888 webserver.cc:460] Webserver started at http://127.21.235.65:40621/ using document root <none> and password file <none>
I20260809 12:57:18.657682 31888 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:18.657810 31888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:18.658130 31888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:18.661043 31888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "6fc6d8601f354906b63f543ca4343317"
format_stamp: "Formatted at 2026-08-09 12:57:18 on dist-test-slave-gp6n"
server_key: "5b733e997d77091948eb18a796961c0c"
server_key_iv: "2f78310d0eca786bc9e5f77ac3b6f95f"
server_key_version: "encryptionkey@0"
I20260809 12:57:18.661819 31888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "6fc6d8601f354906b63f543ca4343317"
format_stamp: "Formatted at 2026-08-09 12:57:18 on dist-test-slave-gp6n"
server_key: "5b733e997d77091948eb18a796961c0c"
server_key_iv: "2f78310d0eca786bc9e5f77ac3b6f95f"
server_key_version: "encryptionkey@0"
I20260809 12:57:18.667208 31888 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 12:57:18.670719 31904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:18.671993 31888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:57:18.672161 31888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "6fc6d8601f354906b63f543ca4343317"
format_stamp: "Formatted at 2026-08-09 12:57:18 on dist-test-slave-gp6n"
server_key: "5b733e997d77091948eb18a796961c0c"
server_key_iv: "2f78310d0eca786bc9e5f77ac3b6f95f"
server_key_version: "encryptionkey@0"
I20260809 12:57:18.672358 31888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:18.699424 31888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:18.703919 31888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:18.704213 31888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:18.705186 31888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:18.706594 31888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:18.706666 31888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:18.706802 31888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:18.706858 31888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:18.739212 31888 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:45479
I20260809 12:57:18.739260 32032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:45479 every 8 connection(s)
I20260809 12:57:18.740525 31888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:57:18.741210 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 31888
I20260809 12:57:18.741451 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:57:18.742517 22445 external_mini_cluster.cc:1428] Setting key 715914b3575d233362c1328dbcbc3626
Aug 09 12:57:18 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280238, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:57:18.763890 32037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:18.743301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:38357 (local address 127.21.235.126:38679)
0809 12:57:18.744006 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:18.744012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:18.745649 (+ 1637us) server_negotiation.cc:407] Connection header received
0809 12:57:18.747618 (+ 1969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:18.747626 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:18.747723 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:18.747863 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:18.749369 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:18.750539 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:18.752081 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:18.752807 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:18.752948 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:18.756605 (+ 3657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:18.756614 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:18.756622 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:18.756668 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:18.760331 (+ 3663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:18.761091 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:18.761099 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:18.761106 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:18.761257 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:18.761675 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:18.761683 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:18.761689 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:18.762105 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:18.762283 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:18.763103 (+ 820us) server_negotiation.cc:299] Negotiation successful
0809 12:57:18.763375 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":288,"threads_started":1}
I20260809 12:57:18.764549 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:18.744073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:38357)
0809 12:57:18.745472 (+ 1399us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:18.745532 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:18.747359 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:18.748049 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:18.748057 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:18.748606 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:18.749193 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:18.749210 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:18.750691 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:18.750694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:18.751932 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:18.751949 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:18.753012 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:18.753017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:18.753206 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:18.753922 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:18.753944 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:18.756394 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:18.760499 (+ 4105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:18.760506 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:18.760519 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:18.760963 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:18.761371 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:18.761374 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:18.761376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:18.761562 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:18.762270 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:18.762276 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:18.762693 (+ 417us) client_negotiation.cc:769] Sending connection context
0809 12:57:18.762949 (+ 256us) client_negotiation.cc:240] Negotiation successful
0809 12:57:18.763376 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1083,"thread_start_us":862,"threads_started":1}
I20260809 12:57:18.766247 32034 heartbeater.cc:347] Connected to a master server at 127.21.235.126:38679
I20260809 12:57:18.766664 32034 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:18.767534 32034 heartbeater.cc:511] Master 127.21.235.126:38679 requested a full tablet report, sending...
I20260809 12:57:18.769743 31818 ts_manager.cc:194] Registered new tserver with Master: 6fc6d8601f354906b63f543ca4343317 (127.21.235.65:45479)
WARNING: no policy specified for kudu/127.21.235.66@KRBTEST.COM; defaulting to no policy
I20260809 12:57:18.771337 31818 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:38357
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
I20260809 12:57:18.806056 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:38679
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:57:19.135218 32041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:19.135547 32041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:19.135725 32041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:19.142762 32041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:19.142906 32041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:19.143086 32041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:57:19.152758 32041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:38679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:19.154016 32041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:19.155314 32041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:19.168712 32047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:19.168744 32046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:19.168885 32051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:19.169461 32041 server_base.cc:1034] running on GCE node
I20260809 12:57:19.170178 32041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:19.171151 32041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:19.172399 32041 hybrid_clock.cc:648] HybridClock initialized: now 1786280239172354 us; error 51 us; skew 500 ppm
I20260809 12:57:19.172716 32041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:19 dist-test-slave-gp6n krb5kdc[31768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280239, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:19.176039 32041 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 12:57:19.177433 32041 webserver.cc:460] Webserver started at http://127.21.235.66:43673/ using document root <none> and password file <none>
I20260809 12:57:19.178112 32041 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:19.178210 32041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:19.178553 32041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:19.181394 32041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "14fa82a5ed5b40888767ae49016e0029"
format_stamp: "Formatted at 2026-08-09 12:57:19 on dist-test-slave-gp6n"
server_key: "82ddce434891bf9a0c7eb4e2ff1e2ca7"
server_key_iv: "abd92423cb9b99a59ecdb88ce2cc2de8"
server_key_version: "encryptionkey@0"
I20260809 12:57:19.182204 32041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "14fa82a5ed5b40888767ae49016e0029"
format_stamp: "Formatted at 2026-08-09 12:57:19 on dist-test-slave-gp6n"
server_key: "82ddce434891bf9a0c7eb4e2ff1e2ca7"
server_key_iv: "abd92423cb9b99a59ecdb88ce2cc2de8"
server_key_version: "encryptionkey@0"
I20260809 12:57:19.187327 32041 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:57:19.190809 32059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:19.192135 32041 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 12:57:19.192332 32041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "14fa82a5ed5b40888767ae49016e0029"
format_stamp: "Formatted at 2026-08-09 12:57:19 on dist-test-slave-gp6n"
server_key: "82ddce434891bf9a0c7eb4e2ff1e2ca7"
server_key_iv: "abd92423cb9b99a59ecdb88ce2cc2de8"
server_key_version: "encryptionkey@0"
I20260809 12:57:19.192536 32041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:19.206036 32041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:19.210390 32041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:19.210839 32041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:19.211767 32041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:19.213124 32041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:19.213199 32041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:19.213328 32041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:19.213409 32041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:19.247246 32041 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:36933
I20260809 12:57:19.247310 32186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:36933 every 8 connection(s)
I20260809 12:57:19.248675 32041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:57:19.251190 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 32041
I20260809 12:57:19.251402 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:57:19.251971 22445 external_mini_cluster.cc:1428] Setting key a8f7e46962bb95b026549ec8d534068d
Aug 09 12:57:19 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280239, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:57:19.272282 32037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:19.251397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:35147 (local address 127.21.235.126:38679)
0809 12:57:19.251626 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:19.251632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:19.253126 (+ 1494us) server_negotiation.cc:407] Connection header received
0809 12:57:19.255766 (+ 2640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:19.255779 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:19.255846 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:19.255928 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:19.258064 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:19.258869 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:19.261321 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:19.261767 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:19.261880 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:19.266205 (+ 4325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:19.266212 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:19.266219 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:19.266266 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:19.268955 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:19.269912 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:19.269916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:19.269918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:19.270047 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:19.270576 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:19.270579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:19.270581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:19.270835 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:19.270966 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:19.271842 (+ 876us) server_negotiation.cc:299] Negotiation successful
0809 12:57:19.272034 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 12:57:19.273828 32189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:19.252059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:35147)
0809 12:57:19.252944 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:19.252996 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:19.255446 (+ 2450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:19.256209 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:19.256225 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:19.257028 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:19.257865 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:19.257893 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:19.259024 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:19.259031 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:19.261153 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:19.261170 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:19.262003 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:19.262013 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:19.262373 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:19.263153 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:19.263187 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:19.265973 (+ 2786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:19.269108 (+ 3135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:19.269119 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:19.269139 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:19.269793 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:19.270163 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:19.270171 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:19.270177 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:19.270457 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:19.270961 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:19.270971 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:19.271424 (+ 453us) client_negotiation.cc:769] Sending connection context
0809 12:57:19.271765 (+ 341us) client_negotiation.cc:240] Negotiation successful
0809 12:57:19.272337 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":336,"threads_started":1}
I20260809 12:57:19.275388 32187 heartbeater.cc:347] Connected to a master server at 127.21.235.126:38679
I20260809 12:57:19.275748 32187 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:19.276671 32187 heartbeater.cc:511] Master 127.21.235.126:38679 requested a full tablet report, sending...
I20260809 12:57:19.278355 31818 ts_manager.cc:194] Registered new tserver with Master: 14fa82a5ed5b40888767ae49016e0029 (127.21.235.66:36933)
I20260809 12:57:19.279273 31818 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:35147
WARNING: no policy specified for kudu/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
I20260809 12:57:19.314487 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:38679
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:57:19.651919 32195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:19.652195 32195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:19.652385 32195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:19.659947 32195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:19.660076 32195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:19.660260 32195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:57:19.670028 32195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:38679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:19.671442 32195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:19.672821 32195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:19.686635 32204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:19.686669 32201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:19.686721 32200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:19.687354 32195 server_base.cc:1034] running on GCE node
I20260809 12:57:19.687976 32195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:19.688886 32195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:19.690124 32195 hybrid_clock.cc:648] HybridClock initialized: now 1786280239690079 us; error 59 us; skew 500 ppm
I20260809 12:57:19.690444 32195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:19 dist-test-slave-gp6n krb5kdc[31768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280239, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:19.693554 32195 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 12:57:19.695118 32195 webserver.cc:460] Webserver started at http://127.21.235.67:41077/ using document root <none> and password file <none>
I20260809 12:57:19.695793 32195 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:19.695957 32195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:19.696270 32195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:19.699177 32195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "bbfeacfec046471c87ca47dabb251c1c"
format_stamp: "Formatted at 2026-08-09 12:57:19 on dist-test-slave-gp6n"
server_key: "3abde28c7ab7a33c34269bf0b92b29c6"
server_key_iv: "14eda096f7d11fae9c611a08920278ba"
server_key_version: "encryptionkey@0"
I20260809 12:57:19.699950 32195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "bbfeacfec046471c87ca47dabb251c1c"
format_stamp: "Formatted at 2026-08-09 12:57:19 on dist-test-slave-gp6n"
server_key: "3abde28c7ab7a33c34269bf0b92b29c6"
server_key_iv: "14eda096f7d11fae9c611a08920278ba"
server_key_version: "encryptionkey@0"
I20260809 12:57:19.704931 32195 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:57:19.708388 32213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:19.709709 32195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:57:19.709875 32195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "bbfeacfec046471c87ca47dabb251c1c"
format_stamp: "Formatted at 2026-08-09 12:57:19 on dist-test-slave-gp6n"
server_key: "3abde28c7ab7a33c34269bf0b92b29c6"
server_key_iv: "14eda096f7d11fae9c611a08920278ba"
server_key_version: "encryptionkey@0"
I20260809 12:57:19.710060 32195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:19.728598 32195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:19.733052 32195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:19.733372 32195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:19.734467 32195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:19.735841 32195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:19.735914 32195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:19.736101 32195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:19.736153 32195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:19.770018 32195 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:45079
I20260809 12:57:19.770071 32337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:45079 every 8 connection(s)
I20260809 12:57:19.771391 32195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:57:19.774538 32034 heartbeater.cc:503] Master 127.21.235.126:38679 was elected leader, sending a full tablet report...
I20260809 12:57:19.778795 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 32195
I20260809 12:57:19.778995 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:57:19.779476 22445 external_mini_cluster.cc:1428] Setting key 1097c8a6509d89161e0cb1da930103ec
Aug 09 12:57:19 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280239, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:57:19.791472 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:19.774091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:42815 (local address 127.21.235.126:38679)
0809 12:57:19.774720 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:19.774727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:19.775823 (+ 1096us) server_negotiation.cc:407] Connection header received
0809 12:57:19.777759 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:19.777765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:19.777859 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:19.777975 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:19.779633 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:19.780651 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:19.782170 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:19.782640 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:19.782772 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:19.785988 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:19.785993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:19.785996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:19.786026 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:19.788401 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:19.789089 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:19.789095 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:19.789099 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:19.789287 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:19.789705 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:19.789712 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:19.789715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:19.790015 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:19.790186 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:19.790882 (+ 696us) server_negotiation.cc:299] Negotiation successful
0809 12:57:19.791105 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":450,"threads_started":1}
I20260809 12:57:19.792335 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:19.774782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.67:42815)
0809 12:57:19.775631 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:19.775668 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:19.777508 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:19.778223 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:19.778231 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:19.778802 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:19.779444 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:19.779461 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:19.780816 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:19.780819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:19.782015 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:19.782028 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:19.782863 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:19.782870 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:19.783091 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:19.783685 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:19.783713 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:19.785782 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:19.788541 (+ 2759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:19.788549 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:19.788563 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:19.788965 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:19.789402 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:19.789407 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:19.789410 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:19.789593 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:19.790149 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:19.790158 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:19.790535 (+ 377us) client_negotiation.cc:769] Sending connection context
0809 12:57:19.790789 (+ 254us) client_negotiation.cc:240] Negotiation successful
0809 12:57:19.791263 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":339,"threads_started":1}
I20260809 12:57:19.793855 32338 heartbeater.cc:347] Connected to a master server at 127.21.235.126:38679
I20260809 12:57:19.794138 32338 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:19.794920 32338 heartbeater.cc:511] Master 127.21.235.126:38679 requested a full tablet report, sending...
I20260809 12:57:19.796274 31817 ts_manager.cc:194] Registered new tserver with Master: bbfeacfec046471c87ca47dabb251c1c (127.21.235.67:45079)
I20260809 12:57:19.797288 31817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:42815
I20260809 12:57:19.806387 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.65.keytab.
Entry for principal oryx/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.65.keytab.
WARNING: no policy specified for oryx/127.21.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.66.keytab.
Entry for principal oryx/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.66.keytab.
WARNING: no policy specified for oryx/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.67.keytab.
Entry for principal oryx/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.67.keytab.
I20260809 12:57:19.967661 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 31888
Aug 09 12:57:19 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 12:57:19.984612 32350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:19.968953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47222 (local address 127.21.235.65:45479)
0809 12:57:19.969653 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:19.969661 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:19.969689 (+ 28us) server_negotiation.cc:407] Connection header received
0809 12:57:19.969752 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:19.969759 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:19.969988 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:19.970120 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:19.971549 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:19.973035 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:19.974312 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:19.974914 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:19.975019 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:19.979192 (+ 4173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:19.979202 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:19.979217 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:19.979258 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:19.981980 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:19.982740 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:19.982746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:19.982749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:19.982875 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:19.983189 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:19.983192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:19.983194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:19.983526 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:19.983750 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:19.984007 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 12:57:19.984234 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":360,"threads_started":1}
I20260809 12:57:19.985888 32028 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:20.069998 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 31888
I20260809 12:57:20.088603 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 32041
Aug 09 12:57:20 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.66@KRBTEST.COM
I20260809 12:57:20.105520 32352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:20.089459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49740 (local address 127.21.235.66:36933)
0809 12:57:20.089997 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:20.090004 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:20.090031 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:57:20.090093 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:20.090099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:20.090320 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:20.090532 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:20.091698 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:20.092723 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:20.093941 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:20.094645 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:20.094736 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:20.099070 (+ 4334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:20.099081 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:20.099095 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:20.099128 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:20.102693 (+ 3565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:20.103317 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:20.103325 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:20.103329 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:20.103475 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:20.103805 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:20.103810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:20.103813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:20.104256 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:20.104559 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:20.104832 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 12:57:20.105093 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":275,"threads_started":1}
I20260809 12:57:20.106979 32182 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:20.189401 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 32041
I20260809 12:57:20.208688 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 32195
Aug 09 12:57:20 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.67@KRBTEST.COM
I20260809 12:57:20.231412 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:20.209729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45798 (local address 127.21.235.67:45079)
0809 12:57:20.210238 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:20.210246 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:20.210275 (+ 29us) server_negotiation.cc:407] Connection header received
0809 12:57:20.210342 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:20.210348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:20.210650 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:20.210793 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:20.212188 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:20.213242 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:20.214846 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:20.215428 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:20.215548 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:20.222426 (+ 6878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:20.222448 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:20.222482 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:20.222539 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:20.227540 (+ 5001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:20.228583 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:20.228595 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:20.228598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:20.228773 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:20.229287 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:20.229295 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:20.229299 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:20.229823 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:20.230196 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:20.230576 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 12:57:20.230886 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":253,"threads_started":1}
I20260809 12:57:20.233202 32333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:20.320746 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 32195
I20260809 12:57:20.338869 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 31783
I20260809 12:57:20.340121 31853 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:20.448005 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 31783
I20260809 12:57:20.471506 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:38679
--webserver_interface=127.21.235.126
--webserver_port=34901
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:38679
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:57:20.803808 32358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:20.804095 32358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:20.804262 32358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:20.811426 32358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:57:20.811529 32358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:20.811625 32358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:20.811724 32358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:57:20.811813 32358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:57:20.821755 32358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:38679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:38679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=34901
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:20.823117 32358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:20.824503 32358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:20.836777 32363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:20.836740 32366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:20.837114 32364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:20.837853 32358 server_base.cc:1034] running on GCE node
I20260809 12:57:20.838750 32358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:20.840196 32358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:20.841408 32358 hybrid_clock.cc:648] HybridClock initialized: now 1786280240841336 us; error 55 us; skew 500 ppm
I20260809 12:57:20.841773 32358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:20 dist-test-slave-gp6n krb5kdc[31768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280240, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:20.845193 32358 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:57:20.846920 32358 webserver.cc:460] Webserver started at http://127.21.235.126:34901/ using document root <none> and password file <none>
I20260809 12:57:20.847662 32358 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:20.847831 32358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:20.853785 32358 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 12:57:20.857038 32374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:20.858589 32358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 12:57:20.858812 32358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "05576becfe7f4c8c9dcb71c8f26844f5"
format_stamp: "Formatted at 2026-08-09 12:57:18 on dist-test-slave-gp6n"
server_key: "42fea83420b21474febd49cbaec236ed"
server_key_iv: "ed3c0555ed81a826d05447b9ca3487b3"
server_key_version: "encryptionkey@0"
I20260809 12:57:20.859870 32358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:20.884599 32358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:20.889048 32358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:20.889335 32358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:20.908619 32358 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:38679
I20260809 12:57:20.908633 32433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:38679 every 8 connection(s)
I20260809 12:57:20.910178 32358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:57:20.916401 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 32358
I20260809 12:57:20.917424 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:45479
--local_ip_for_outbound_sockets=127.21.235.65
--tserver_master_addrs=127.21.235.126:38679
--webserver_port=40621
--webserver_interface=127.21.235.65
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.235.65
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:57:20.918381 32434 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:57:20.923624 32434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5: Bootstrap starting.
I20260809 12:57:20.944159 32434 log.cc:824] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5: Log is configured to *not* fsync() on all Append() calls
I20260809 12:57:20.953315 32434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:57:20.953883 32434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5: Bootstrap complete.
I20260809 12:57:20.963228 32434 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05576becfe7f4c8c9dcb71c8f26844f5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 38679 } }
I20260809 12:57:20.964198 32434 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05576becfe7f4c8c9dcb71c8f26844f5, State: Initialized, Role: FOLLOWER
I20260809 12:57:20.964926 32434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05576becfe7f4c8c9dcb71c8f26844f5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 38679 } }
I20260809 12:57:20.965169 32434 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:57:20.965281 32434 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:57:20.965456 32434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 12:57:20.969444 32434 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05576becfe7f4c8c9dcb71c8f26844f5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 38679 } }
I20260809 12:57:20.969947 32434 leader_election.cc:302] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05576becfe7f4c8c9dcb71c8f26844f5; no voters:
I20260809 12:57:20.970597 32434 leader_election.cc:288] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 12:57:20.970746 32439 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 12:57:20.972276 32439 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [term 2 LEADER]: Becoming Leader. State: Replica: 05576becfe7f4c8c9dcb71c8f26844f5, State: Running, Role: LEADER
I20260809 12:57:20.972846 32439 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05576becfe7f4c8c9dcb71c8f26844f5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 38679 } }
I20260809 12:57:20.973109 32434 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:57:20.976112 32440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "05576becfe7f4c8c9dcb71c8f26844f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05576becfe7f4c8c9dcb71c8f26844f5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 38679 } } }
I20260809 12:57:20.976559 32440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:20.979624 32446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:57:20.979542 32441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05576becfe7f4c8c9dcb71c8f26844f5. Latest consensus state: current_term: 2 leader_uuid: "05576becfe7f4c8c9dcb71c8f26844f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05576becfe7f4c8c9dcb71c8f26844f5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 38679 } } }
I20260809 12:57:20.979956 32441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5 [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:20.985419 32446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:57:20.986863 32446 catalog_manager.cc:1259] Loaded cluster ID: e31d88ce861443e5bcb9c0beb33530ab
I20260809 12:57:20.987025 32446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:57:20.990986 32446 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:57:20.992440 32446 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 05576becfe7f4c8c9dcb71c8f26844f5: Loaded TSK: 0
I20260809 12:57:20.994033 32446 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 12:57:21.270503 32436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:21.270861 32436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:21.271031 32436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:21.278373 32436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:21.278576 32436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:21.278750 32436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:57:21.288450 32436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.65.keytab
--principal=oryx/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:45479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=40621
--tserver_master_addrs=127.21.235.126:38679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:21.289772 32436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:21.291172 32436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:21.311769 32466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:21.312292 32468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:21.312487 32465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:21.313516 32436 server_base.cc:1034] running on GCE node
I20260809 12:57:21.314159 32436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:21.315317 32436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:21.316571 32436 hybrid_clock.cc:648] HybridClock initialized: now 1786280241316506 us; error 58 us; skew 500 ppm
I20260809 12:57:21.316911 32436 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:21 dist-test-slave-gp6n krb5kdc[31768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280241, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:21.324685 32436 init.cc:377] Logged in from keytab as oryx/127.21.235.65@KRBTEST.COM (short username oryx)
I20260809 12:57:21.326498 32436 webserver.cc:460] Webserver started at http://127.21.235.65:40621/ using document root <none> and password file <none>
I20260809 12:57:21.327271 32436 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:21.327447 32436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:21.333039 32436 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 12:57:21.335908 32475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:21.337364 32436 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:57:21.337543 32436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "6fc6d8601f354906b63f543ca4343317"
format_stamp: "Formatted at 2026-08-09 12:57:18 on dist-test-slave-gp6n"
server_key: "5b733e997d77091948eb18a796961c0c"
server_key_iv: "2f78310d0eca786bc9e5f77ac3b6f95f"
server_key_version: "encryptionkey@0"
I20260809 12:57:21.338579 32436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:21.361860 32436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:21.367287 32436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:21.367575 32436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:21.368614 32436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:21.370354 32436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:21.370512 32436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260809 12:57:21.370636 32436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:21.370721 32436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:21.409358 32436 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:45479
I20260809 12:57:21.409471 32604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:45479 every 8 connection(s)
I20260809 12:57:21.410950 32436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:57:21.416879 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 32436
I20260809 12:57:21.417603 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:36933
--local_ip_for_outbound_sockets=127.21.235.66
--tserver_master_addrs=127.21.235.126:38679
--webserver_port=43673
--webserver_interface=127.21.235.66
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.235.66
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:57:21 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:21 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:21.431245 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:21.414546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:32785)
0809 12:57:21.415377 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:21.415432 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:21.417292 (+ 1860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:21.419092 (+ 1800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:21.419099 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:21.419669 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:21.420245 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:21.420262 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.422658 (+ 2396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.422662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:21.423970 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:21.423983 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.425511 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.425517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:21.425711 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:21.426518 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:21.426550 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:21.430220 (+ 3670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":364,"threads_started":1}
W20260809 12:57:21.431962 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:21.414478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:32785 (local address 127.21.235.126:38679)
0809 12:57:21.415320 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:21.415333 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:21.415563 (+ 230us) server_negotiation.cc:407] Connection header received
0809 12:57:21.417964 (+ 2401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:21.417998 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:21.418517 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:21.418951 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:21.420466 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.422471 (+ 2005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.424132 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.425276 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.425479 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:21.430671 (+ 5192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.65:32785: BlockingRecv error: recv got EOF from 127.21.235.65:32785 (error 108)
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":312,"threads_started":1}
W20260809 12:57:21.432070 32605 heartbeater.cc:644] Failed to heartbeat to 127.21.235.126:38679 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.235.126:38679: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Aug 09 12:57:21 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:21 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:21.442057 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:21.432689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:50483)
0809 12:57:21.432972 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:21.432992 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:21.433093 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:21.433455 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:21.433459 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:21.433758 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:21.433909 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:21.433916 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.435542 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.435546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:21.436465 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:21.436476 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.437500 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.437505 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:21.437627 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:21.438280 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:21.438299 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:21.441701 (+ 3402us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158}
W20260809 12:57:21.442297 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:21.432963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:50483 (local address 127.21.235.126:38679)
0809 12:57:21.433147 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:21.433154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:21.433175 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:57:21.433230 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:21.433237 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:21.433315 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:21.433418 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:21.434051 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.435379 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.436615 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.437344 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.437496 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:21.442002 (+ 4506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.65:50483: BlockingRecv error: recv got EOF from 127.21.235.65:50483 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 12:57:21 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:21 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:21.452761 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:21.442956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:37703)
0809 12:57:21.443138 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:21.443160 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:21.443275 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:21.443649 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:21.443657 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:21.444123 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:21.444360 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:21.444376 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.445523 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.445531 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:21.447338 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:21.447356 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.448203 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.448209 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:21.448421 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:21.449061 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:21.449084 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:21.452444 (+ 3360us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 12:57:21.453066 32605 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 12:57:21.453217 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:21.443036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:37703 (local address 127.21.235.126:38679)
0809 12:57:21.443224 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:21.443229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:21.443247 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:57:21.443406 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:21.443412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:21.443497 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:21.443583 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:21.444521 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.445366 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.447510 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.448047 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.448152 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:21.452655 (+ 4503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.65:37703: BlockingRecv error: recv got EOF from 127.21.235.65:37703 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 12:57:21.758493 32610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:21.758841 32610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:21.759029 32610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:21.766477 32610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:21.766621 32610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:21.766804 32610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:57:21.776539 32610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.66.keytab
--principal=oryx/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:36933
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=43673
--tserver_master_addrs=127.21.235.126:38679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:21.777902 32610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:21.779234 32610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:21.792476 32619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:21.792600 32616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:21.792608 32615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:21.793241 32610 server_base.cc:1034] running on GCE node
I20260809 12:57:21.793887 32610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:21.794983 32610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:21.796238 32610 hybrid_clock.cc:648] HybridClock initialized: now 1786280241796196 us; error 38 us; skew 500 ppm
I20260809 12:57:21.796603 32610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:21 dist-test-slave-gp6n krb5kdc[31768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280241, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:21.799882 32610 init.cc:377] Logged in from keytab as oryx/127.21.235.66@KRBTEST.COM (short username oryx)
I20260809 12:57:21.801402 32610 webserver.cc:460] Webserver started at http://127.21.235.66:43673/ using document root <none> and password file <none>
I20260809 12:57:21.802145 32610 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:21.802305 32610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:21.807744 32610 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:57:21.810309 32630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:21.811625 32610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:57:21.811802 32610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "14fa82a5ed5b40888767ae49016e0029"
format_stamp: "Formatted at 2026-08-09 12:57:19 on dist-test-slave-gp6n"
server_key: "82ddce434891bf9a0c7eb4e2ff1e2ca7"
server_key_iv: "abd92423cb9b99a59ecdb88ce2cc2de8"
server_key_version: "encryptionkey@0"
I20260809 12:57:21.812852 32610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:21.834524 32610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:21.839043 32610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:21.839272 32610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:21.840200 32610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:21.841504 32610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:21.841590 32610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:21.841665 32610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:21.841720 32610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:21.874703 32610 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:36933
I20260809 12:57:21.874788 32759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:36933 every 8 connection(s)
I20260809 12:57:21.876191 32610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:57:21.877110 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 32610
I20260809 12:57:21.877784 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:45079
--local_ip_for_outbound_sockets=127.21.235.67
--tserver_master_addrs=127.21.235.126:38679
--webserver_port=41077
--webserver_interface=127.21.235.67
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.235.67
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:57:21 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:21 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 12:57:21.894238 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:21.878952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:53953 (local address 127.21.235.126:38679)
0809 12:57:21.879170 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:21.879176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:21.880496 (+ 1320us) server_negotiation.cc:407] Connection header received
0809 12:57:21.882337 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:21.882345 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:21.882467 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:21.882586 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:21.884176 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.885288 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.887020 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.887468 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.887564 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:21.894007 (+ 6443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.66:53953: BlockingRecv error: recv got EOF from 127.21.235.66:53953 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 12:57:21.895237 32762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:21.879507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:53953)
0809 12:57:21.880310 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:21.880366 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:21.882058 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:21.882829 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:21.882838 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:21.883432 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:21.883986 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:21.884004 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.885519 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.885523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:21.886867 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:21.886880 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.887720 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.887728 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:21.888093 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:21.888844 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:21.888884 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:21.893780 (+ 4896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":339,"threads_started":1}
W20260809 12:57:21.895994 32760 heartbeater.cc:644] Failed to heartbeat to 127.21.235.126:38679 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.235.126:38679: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Aug 09 12:57:21 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:21 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 12:57:21.906188 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:21.896720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:43247 (local address 127.21.235.126:38679)
0809 12:57:21.896885 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:21.896889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:21.896905 (+ 16us) server_negotiation.cc:407] Connection header received
0809 12:57:21.897042 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:21.897045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:21.897098 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:21.897174 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:21.898031 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.898790 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.900758 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.901151 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.901254 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:21.906020 (+ 4766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.66:43247: BlockingRecv error: recv got EOF from 127.21.235.66:43247 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 12:57:21.906203 32762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:21.896568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:43247)
0809 12:57:21.896773 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:21.896795 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:21.896908 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:21.897232 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:21.897239 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:21.897681 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:21.897894 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:21.897908 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.898930 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.898939 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:21.900619 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:21.900636 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.901311 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.901319 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:21.901552 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:21.902156 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:21.902185 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:21.905857 (+ 3672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
Aug 09 12:57:21 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:21 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:21.917068 32762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:21.906879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:45127)
0809 12:57:21.907165 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:21.907185 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:21.907275 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:21.907682 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:21.907685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:21.908154 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:21.908342 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:21.908353 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.909993 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.910001 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:21.911469 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:21.911483 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.912449 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.912455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:21.912624 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:21.913313 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:21.913334 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:21.916677 (+ 3343us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183}
W20260809 12:57:21.917137 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:21.907122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:45127 (local address 127.21.235.126:38679)
0809 12:57:21.907327 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:21.907335 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:21.907355 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:57:21.907437 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:21.907445 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:21.907539 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:21.907649 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:21.908495 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.909809 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.911615 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:21.912304 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:21.912431 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:21.916922 (+ 4491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.66:45127: BlockingRecv error: recv got EOF from 127.21.235.66:45127 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 12:57:21.917716 32760 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 12:57:22.231576 32763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:22.231871 32763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:22.232064 32763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:22.239862 32763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:22.239959 32763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:22.240149 32763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:57:22.249882 32763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/krb5kdc/oryx_127.21.235.67.keytab
--principal=oryx/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:45079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=41077
--tserver_master_addrs=127.21.235.126:38679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:22.251171 32763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:22.252489 32763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:22.266394 300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:22.266611 303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:22.266394 301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:22.267170 32763 server_base.cc:1034] running on GCE node
I20260809 12:57:22.268033 32763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:22.269040 32763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:22.270288 32763 hybrid_clock.cc:648] HybridClock initialized: now 1786280242270205 us; error 74 us; skew 500 ppm
I20260809 12:57:22.270649 32763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:22 dist-test-slave-gp6n krb5kdc[31768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280242, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:22.274111 32763 init.cc:377] Logged in from keytab as oryx/127.21.235.67@KRBTEST.COM (short username oryx)
I20260809 12:57:22.275724 32763 webserver.cc:460] Webserver started at http://127.21.235.67:41077/ using document root <none> and password file <none>
I20260809 12:57:22.276440 32763 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:22.276597 32763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:22.282042 32763 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:57:22.284938 311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:22.286337 32763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:57:22.286592 32763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "bbfeacfec046471c87ca47dabb251c1c"
format_stamp: "Formatted at 2026-08-09 12:57:19 on dist-test-slave-gp6n"
server_key: "3abde28c7ab7a33c34269bf0b92b29c6"
server_key_iv: "14eda096f7d11fae9c611a08920278ba"
server_key_version: "encryptionkey@0"
I20260809 12:57:22.287601 32763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:22.300264 32763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:22.304524 32763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:22.304798 32763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:22.305778 32763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:22.307132 32763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:22.307255 32763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:22.307387 32763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:22.307483 32763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:22.348852 32763 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:45079
I20260809 12:57:22.348944 445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:45079 every 8 connection(s)
I20260809 12:57:22.350432 32763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:57:22.353478 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 32763
Aug 09 12:57:22 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:22 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 12:57:22.367726 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:22.353089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:55227 (local address 127.21.235.126:38679)
0809 12:57:22.353286 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:22.353291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:22.354933 (+ 1642us) server_negotiation.cc:407] Connection header received
0809 12:57:22.357178 (+ 2245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:22.357187 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:22.357327 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:22.357441 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:22.359180 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.359998 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.361699 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.362218 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.362347 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:22.367432 (+ 5085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.67:55227: BlockingRecv error: recv got EOF from 127.21.235.67:55227 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":57}
I20260809 12:57:22.368324 448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:22.353804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.67:55227)
0809 12:57:22.354748 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:22.354804 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:22.356889 (+ 2085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:22.357741 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:22.357752 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:22.358352 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:22.358999 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:22.359016 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.360153 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.360158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:22.361552 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:22.361566 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.362494 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.362500 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:22.362695 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:22.363310 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:22.363365 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:22.367144 (+ 3779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":398,"threads_started":1}
W20260809 12:57:22.369109 446 heartbeater.cc:644] Failed to heartbeat to 127.21.235.126:38679 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.235.126:38679: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
I20260809 12:57:22.372983 450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:22.354610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49202 (local address 127.21.235.126:38679)
0809 12:57:22.357490 (+ 2880us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:22.357496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:22.357514 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:57:22.357567 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:22.357571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:22.357635 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:22.357753 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:22.359356 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.360294 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.361543 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.362066 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.362159 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:22.365668 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:22.365679 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:22.365701 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:22.365750 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:22.369421 (+ 3671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:22.370220 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:22.370227 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:22.370233 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:22.370467 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:22.370844 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:22.370849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:22.370857 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:22.371344 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:22.371690 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:22.372116 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 12:57:22.372515 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":2745,"thread_start_us":2628,"threads_started":1}
Aug 09 12:57:22 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:22 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:22.378654 448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:22.369739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.67:36659)
0809 12:57:22.369914 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:22.369934 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:22.370030 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:22.370384 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:22.370388 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:22.370729 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:22.370870 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:22.370878 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.371977 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.371982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:22.373363 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:22.373381 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.374554 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.374560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:22.374768 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:22.375288 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:22.375303 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:22.378451 (+ 3148us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 12:57:22.378901 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:22.369809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:36659 (local address 127.21.235.126:38679)
0809 12:57:22.370000 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:22.370005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:22.370021 (+ 16us) server_negotiation.cc:407] Connection header received
0809 12:57:22.370157 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:22.370162 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:22.370230 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:22.370377 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:22.371008 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.371836 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.373555 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.374173 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.374296 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:22.378643 (+ 4347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.67:36659: BlockingRecv error: recv got EOF from 127.21.235.67:36659 (error 108)
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":56}
Aug 09 12:57:22 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:22 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:22.387845 448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:22.379443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.67:42125)
0809 12:57:22.379675 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:22.379698 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:22.379820 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:22.380141 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:22.380146 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:22.380609 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:22.380866 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:22.380877 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.381922 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.381928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:22.383139 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:22.383152 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.383898 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.383904 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:22.384131 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:22.384715 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:22.384742 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:22.387585 (+ 2843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":78}
W20260809 12:57:22.388139 446 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 12:57:22.388309 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:22.379542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:42125 (local address 127.21.235.126:38679)
0809 12:57:22.379802 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:22.379808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:22.379828 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:57:22.379929 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:22.379934 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:22.380010 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:22.380122 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:22.381010 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.381780 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.383300 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.383748 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.383888 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:22.387868 (+ 3980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.67:42125: BlockingRecv error: recv got EOF from 127.21.235.67:42125 (error 108)
Metrics: {"server-negotiator.queue_time_us":145}
Aug 09 12:57:22 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:22 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:22.462625 451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:22.453654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:50083)
0809 12:57:22.454215 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:22.454230 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:22.454459 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:22.454800 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:22.454805 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:22.455325 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:22.455515 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:22.455524 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.456559 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.456565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:22.457533 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:22.457545 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.458314 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.458323 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:22.458517 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:22.459149 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:22.459167 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:22.462241 (+ 3074us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":302,"threads_started":1}
W20260809 12:57:22.462711 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:22.453784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:50083 (local address 127.21.235.126:38679)
0809 12:57:22.454046 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:22.454054 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:22.454350 (+ 296us) server_negotiation.cc:407] Connection header received
0809 12:57:22.454601 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:22.454605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:22.454665 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:22.454766 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:22.455659 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.456410 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.457677 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.458111 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.458254 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:22.462487 (+ 4233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.65:50083: BlockingRecv error: recv got EOF from 127.21.235.65:50083 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 09 12:57:22 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:22 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:22.928027 452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:22.918404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:46709)
0809 12:57:22.918967 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:22.918986 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:22.919124 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:22.919543 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:22.919547 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:22.920035 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:22.920229 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:22.920238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.921360 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.921364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:22.922707 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:22.922723 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.923502 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.923507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:22.923636 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:22.924264 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:22.924318 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:22.927760 (+ 3442us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":281,"threads_started":1}
W20260809 12:57:22.928274 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:22.918472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:46709 (local address 127.21.235.126:38679)
0809 12:57:22.918695 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:22.918705 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:22.919078 (+ 373us) server_negotiation.cc:407] Connection header received
0809 12:57:22.919291 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:22.919298 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:22.919372 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:22.919472 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:22.920430 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.921170 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.922864 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:22.923307 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:22.923413 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:22.927993 (+ 4580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.66:46709: BlockingRecv error: recv got EOF from 127.21.235.66:46709 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 09 12:57:23 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:23 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:23.402473 453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:23.388725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.67:34657)
0809 12:57:23.389248 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:23.389276 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:23.389449 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:23.389783 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:23.389787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:23.390284 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:23.390555 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:23.390566 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:23.391595 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:23.391604 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:23.392658 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:23.392670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:23.393535 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:23.393541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:23.393721 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:23.398794 (+ 5073us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:23.398816 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:23.402188 (+ 3372us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":284,"threads_started":1}
W20260809 12:57:23.402791 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:23.388787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:34657 (local address 127.21.235.126:38679)
0809 12:57:23.389032 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:23.389041 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:23.389367 (+ 326us) server_negotiation.cc:407] Connection header received
0809 12:57:23.389576 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:23.389581 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:23.389649 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:23.389752 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:23.390692 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:23.391444 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:23.392800 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:23.393349 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:23.393483 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:23.402513 (+ 9030us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.67:34657: BlockingRecv error: recv got EOF from 127.21.235.67:34657 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 12:57:23 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:23 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:23.472224 455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:23.463518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:51195)
0809 12:57:23.464134 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:23.464151 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:23.464277 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:23.464567 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:23.464570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:23.465034 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:23.465211 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:23.465220 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:23.466253 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:23.466256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:23.467234 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:23.467246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:23.467904 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:23.467908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:23.468034 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:23.468595 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:23.468612 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:23.471869 (+ 3257us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":336,"threads_started":1}
W20260809 12:57:23.472333 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:23.463623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:51195 (local address 127.21.235.126:38679)
0809 12:57:23.463874 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:23.463884 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:23.464223 (+ 339us) server_negotiation.cc:407] Connection header received
0809 12:57:23.464379 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:23.464382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:23.464445 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:23.464528 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:23.465364 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:23.466137 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:23.467359 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:23.467771 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:23.467874 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:23.472073 (+ 4199us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.65:51195: BlockingRecv error: recv got EOF from 127.21.235.65:51195 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 12:57:23 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:23 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:23.937690 457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:23.929050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:33371)
0809 12:57:23.929709 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:23.929727 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:23.929915 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:23.930309 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:23.930316 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:23.930782 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:23.930966 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:23.930976 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:23.932046 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:23.932052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:23.932949 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:23.932960 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:23.933679 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:23.933685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:23.933856 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:23.934512 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:23.934531 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:23.937453 (+ 2922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":346,"threads_started":1}
W20260809 12:57:23.937872 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:23.929230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:33371 (local address 127.21.235.126:38679)
0809 12:57:23.929478 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:23.929487 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:23.929855 (+ 368us) server_negotiation.cc:407] Connection header received
0809 12:57:23.930077 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:23.930081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:23.930143 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:23.930260 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:23.931115 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:23.931904 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:23.933091 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:23.933520 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:23.933659 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:23.937653 (+ 3994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.66:33371: BlockingRecv error: recv got EOF from 127.21.235.66:33371 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 09 12:57:24 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:24 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:24.412936 458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:24.403402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.67:42279)
0809 12:57:24.403991 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:24.404008 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:24.404174 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:24.404547 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:24.404551 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:24.404977 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:24.405149 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:24.405158 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:24.406209 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:24.406215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:24.407167 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:24.407178 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:24.407918 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:24.407923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:24.408096 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:24.408765 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:24.408787 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:24.412580 (+ 3793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":318,"threads_started":1}
W20260809 12:57:24.413239 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:24.403646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:42279 (local address 127.21.235.126:38679)
0809 12:57:24.403932 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:24.403939 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:24.404094 (+ 155us) server_negotiation.cc:407] Connection header received
0809 12:57:24.404322 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:24.404326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:24.404391 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:24.404503 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:24.405298 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:24.406060 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:24.407307 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:24.407758 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:24.407907 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:24.412918 (+ 5011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.67:42279: BlockingRecv error: recv got EOF from 127.21.235.67:42279 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 09 12:57:24 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:24 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:24.482141 459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:24.473244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:35987)
0809 12:57:24.473798 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:24.473815 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:24.473970 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:24.474351 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:24.474355 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:24.474837 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:24.475028 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:24.475037 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:24.476085 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:24.476092 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:24.477081 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:24.477094 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:24.477811 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:24.477817 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:24.477998 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:24.478717 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:24.478738 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:24.481877 (+ 3139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":324,"threads_started":1}
W20260809 12:57:24.482308 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:24.473330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:35987 (local address 127.21.235.126:38679)
0809 12:57:24.473570 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:24.473578 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:24.473898 (+ 320us) server_negotiation.cc:407] Connection header received
0809 12:57:24.474156 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:24.474160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:24.474220 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:24.474318 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:24.475176 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:24.475945 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:24.477222 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:24.477659 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:24.477787 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:24.482066 (+ 4279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.65:35987: BlockingRecv error: recv got EOF from 127.21.235.65:35987 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 12:57:24 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:24 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:24.946871 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:24.938796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:60731)
0809 12:57:24.939363 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:24.939380 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:24.939495 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:24.939745 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:24.939748 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:24.940114 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:24.940283 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:24.940292 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:24.941271 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:24.941274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:24.942203 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:24.942214 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:24.943010 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:24.943014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:24.943136 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:24.943795 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:24.943812 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:24.946624 (+ 2812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":299,"threads_started":1}
W20260809 12:57:24.947063 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:24.938845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:60731 (local address 127.21.235.126:38679)
0809 12:57:24.939013 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:24.939018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:24.939450 (+ 432us) server_negotiation.cc:407] Connection header received
0809 12:57:24.939588 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:24.939592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:24.939652 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:24.939730 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:24.940406 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:24.941160 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:24.942322 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:24.942834 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:24.943002 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:24.946814 (+ 3812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.66:60731: BlockingRecv error: recv got EOF from 127.21.235.66:60731 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 12:57:25 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:25 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:25.422770 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:25.413792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.67:49359)
0809 12:57:25.414326 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:25.414343 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:25.414525 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:25.414844 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:25.414847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:25.415299 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:25.415477 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:25.415487 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:25.416484 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:25.416488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:25.417550 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:25.417562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:25.418190 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:25.418194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:25.418313 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:25.418973 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:25.418990 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:25.422349 (+ 3359us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":296,"threads_started":1}
W20260809 12:57:25.422889 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:25.413956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:49359 (local address 127.21.235.126:38679)
0809 12:57:25.414152 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:25.414156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:25.414440 (+ 284us) server_negotiation.cc:407] Connection header received
0809 12:57:25.414631 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:25.414635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:25.414707 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:25.414793 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:25.415636 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:25.416365 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:25.417674 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:25.418072 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:25.418164 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:25.422607 (+ 4443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.67:49359: BlockingRecv error: recv got EOF from 127.21.235.67:49359 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 12:57:25 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:25 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:25.492331 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:25.483084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:34123)
0809 12:57:25.483628 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:25.483646 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:25.483817 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:25.484192 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:25.484197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:25.484656 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:25.484825 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:25.484835 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:25.486004 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:25.486018 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:25.487177 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:25.487190 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:25.487983 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:25.487989 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:25.488176 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:25.488874 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:25.488896 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:25.491957 (+ 3061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":313,"threads_started":1}
W20260809 12:57:25.492574 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:25.483146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:34123 (local address 127.21.235.126:38679)
0809 12:57:25.483395 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:25.483404 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:25.483738 (+ 334us) server_negotiation.cc:407] Connection header received
0809 12:57:25.483952 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:25.483963 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:25.484029 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:25.484140 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:25.484990 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:25.485839 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:25.487336 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:25.487807 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:25.487971 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:25.492262 (+ 4291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.65:34123: BlockingRecv error: recv got EOF from 127.21.235.65:34123 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 12:57:25 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:25 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:25.956321 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:25.947854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:60931)
0809 12:57:25.948393 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:25.948410 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:25.948528 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:25.948846 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:25.948850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:25.949237 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:25.949398 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:25.949407 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:25.950480 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:25.950484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:25.951450 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:25.951462 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:25.952125 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:25.952129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:25.952250 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:25.952932 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:25.952950 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:25.956070 (+ 3120us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":297,"threads_started":1}
W20260809 12:57:25.956525 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:25.947924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:60931 (local address 127.21.235.126:38679)
0809 12:57:25.948122 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:25.948127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:25.948490 (+ 363us) server_negotiation.cc:407] Connection header received
0809 12:57:25.948629 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:25.948633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:25.948696 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:25.948781 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:25.949549 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:25.950313 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:25.951584 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:25.952005 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:25.952099 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:25.956295 (+ 4196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.66:60931: BlockingRecv error: recv got EOF from 127.21.235.66:60931 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 09 12:57:26 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:26 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:26.433595 464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:26.423881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.67:58851)
0809 12:57:26.424356 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:26.424374 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:26.424509 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:26.424840 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:26.424843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:26.425336 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:26.425571 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:26.425585 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:26.427157 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:26.427166 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:26.428673 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:26.428686 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:26.429430 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:26.429434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:26.429564 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:26.430187 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:26.430207 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:26.433349 (+ 3142us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":263,"threads_started":1}
W20260809 12:57:26.433782 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:26.423925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:58851 (local address 127.21.235.126:38679)
0809 12:57:26.424181 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:26.424188 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:26.424470 (+ 282us) server_negotiation.cc:407] Connection header received
0809 12:57:26.424614 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:26.424618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:26.424693 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:26.424779 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:26.425757 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:26.427017 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:26.428806 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:26.429242 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:26.429353 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:26.433548 (+ 4195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.67:58851: BlockingRecv error: recv got EOF from 127.21.235.67:58851 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 12:57:26 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:26 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:26.502838 465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:26.493687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:37279)
0809 12:57:26.494256 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:26.494272 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:26.494470 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:26.494817 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:26.494822 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:26.495304 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:26.495461 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:26.495470 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:26.496536 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:26.496543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:26.497468 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:26.497480 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:26.498216 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:26.498221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:26.498424 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:26.499081 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:26.499098 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:26.502537 (+ 3439us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":292,"threads_started":1}
W20260809 12:57:26.502871 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:26.493756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:37279 (local address 127.21.235.126:38679)
0809 12:57:26.493986 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:26.493992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:26.494358 (+ 366us) server_negotiation.cc:407] Connection header received
0809 12:57:26.494604 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:26.494609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:26.494678 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:26.494782 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:26.495633 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:26.496388 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:26.497624 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:26.498069 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:26.498206 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:26.502701 (+ 4495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.65:37279: BlockingRecv error: recv got EOF from 127.21.235.65:37279 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 12:57:26 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:26 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:26.966244 466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:26.957184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:40127)
0809 12:57:26.957791 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:26.957810 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:26.957978 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:26.958343 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:26.958347 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:26.958789 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:26.958959 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:26.958967 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:26.960035 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:26.960044 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:26.960988 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:26.960999 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:26.961754 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:26.961760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:26.961937 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:26.962647 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:26.962665 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:26.965989 (+ 3324us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":321,"threads_started":1}
W20260809 12:57:26.966470 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:26.957423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:40127 (local address 127.21.235.126:38679)
0809 12:57:26.957671 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:26.957679 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:26.957908 (+ 229us) server_negotiation.cc:407] Connection header received
0809 12:57:26.958104 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:26.958110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:26.958187 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:26.958290 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:26.959107 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:26.959885 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:26.961140 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:26.961600 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:26.961731 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:26.966164 (+ 4433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.66:40127: BlockingRecv error: recv got EOF from 127.21.235.66:40127 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 09 12:57:27 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:27 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:27.442914 467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:27.434604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.67:50851)
0809 12:57:27.435109 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:27.435126 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:27.435261 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:27.435623 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:27.435626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:27.436029 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:27.436194 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:27.436203 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:27.437173 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:27.437176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:27.438082 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:27.438094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:27.438860 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:27.438864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:27.438988 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:27.439586 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:27.439603 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:27.442615 (+ 3012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":227,"threads_started":1}
W20260809 12:57:27.443080 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:27.434679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:50851 (local address 127.21.235.126:38679)
0809 12:57:27.434978 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:27.434984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:27.435201 (+ 217us) server_negotiation.cc:407] Connection header received
0809 12:57:27.435430 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:27.435435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:27.435521 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:27.435603 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:27.436324 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:27.437047 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:27.438201 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:27.438732 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:27.438826 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:27.442818 (+ 3992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.67:50851: BlockingRecv error: recv got EOF from 127.21.235.67:50851 (error 108)
Metrics: {"server-negotiator.queue_time_us":165}
Aug 09 12:57:27 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:27 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:27.512540 468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:27.503882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:37317)
0809 12:57:27.504450 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:27.504467 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:27.504614 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:27.504956 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:27.504964 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:27.505467 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:27.505630 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:27.505639 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:27.506754 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:27.506760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:27.507672 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:27.507684 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:27.508419 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:27.508424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:27.508617 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:27.509205 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:27.509222 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:27.512187 (+ 2965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":312,"threads_started":1}
W20260809 12:57:27.512661 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:27.503943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:37317 (local address 127.21.235.126:38679)
0809 12:57:27.504143 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:27.504149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:27.504559 (+ 410us) server_negotiation.cc:407] Connection header received
0809 12:57:27.504733 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:27.504737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:27.504801 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:27.504912 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:27.505781 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:27.506614 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:27.507824 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:27.508250 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:27.508386 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:27.512407 (+ 4021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.65:37317: BlockingRecv error: recv got EOF from 127.21.235.65:37317 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 12:57:27 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:27 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:27.976015 470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:27.967266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:35911)
0809 12:57:27.967838 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:27.967862 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:27.968022 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:27.968360 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:27.968364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:27.968787 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:27.968955 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:27.968963 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:27.970330 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:27.970334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:27.971516 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:27.971529 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:27.972173 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:27.972177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:27.972293 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:27.972874 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:27.972891 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:27.975791 (+ 2900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":320,"threads_started":1}
W20260809 12:57:27.976325 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:27.967368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:35911 (local address 127.21.235.126:38679)
0809 12:57:27.967642 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:27.967652 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:27.967951 (+ 299us) server_negotiation.cc:407] Connection header received
0809 12:57:27.968151 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:27.968160 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:27.968219 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:27.968322 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:27.969103 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:27.970166 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:27.971653 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:27.972043 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:27.972139 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:27.976039 (+ 3900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.66:35911: BlockingRecv error: recv got EOF from 127.21.235.66:35911 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 09 12:57:28 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:28 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:28.456931 471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:28.443874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.67:46535)
0809 12:57:28.444445 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:28.444462 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:28.444621 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:28.444985 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:28.444990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:28.445421 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:28.445572 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:28.445581 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:28.446733 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:28.446741 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:28.447670 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:28.447681 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:28.448504 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:28.448510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:28.448685 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:28.453738 (+ 5053us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:28.453757 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:28.456600 (+ 2843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":314,"threads_started":1}
W20260809 12:57:28.457073 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:28.443960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:46535 (local address 127.21.235.126:38679)
0809 12:57:28.444224 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:28.444233 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:28.444547 (+ 314us) server_negotiation.cc:407] Connection header received
0809 12:57:28.444742 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:28.444747 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:28.444841 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:28.444945 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:28.445723 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:28.446592 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:28.447814 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:28.448314 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:28.448486 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:28.456810 (+ 8324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.67:46535: BlockingRecv error: recv got EOF from 127.21.235.67:46535 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 09 12:57:28 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:28 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:28.522586 472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:28.513704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:40899)
0809 12:57:28.514297 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:28.514314 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:28.514497 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:28.514859 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:28.514863 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:28.515348 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:28.515548 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:28.515558 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:28.516558 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:28.516562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:28.517767 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:28.517779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:28.518573 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:28.518577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:28.518688 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:28.519332 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:28.519350 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:28.522206 (+ 2856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":279,"threads_started":1}
W20260809 12:57:28.522780 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:28.513777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:40899 (local address 127.21.235.126:38679)
0809 12:57:28.514016 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:28.514033 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:28.514443 (+ 410us) server_negotiation.cc:407] Connection header received
0809 12:57:28.514606 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:28.514610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:28.514675 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:28.514758 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:28.515700 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:28.516425 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:28.517904 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:28.518350 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:28.518564 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:28.522397 (+ 3833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.65:40899: BlockingRecv error: recv got EOF from 127.21.235.65:40899 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 09 12:57:28 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:28 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:28.985849 475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:28.977047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:60875)
0809 12:57:28.977731 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:28.977748 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:28.977900 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:28.978277 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:28.978281 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:28.978746 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:28.978914 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:28.978922 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:28.980001 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:28.980006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:28.980944 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:28.980955 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:28.981637 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:28.981643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:28.981824 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:28.982447 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:28.982465 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:28.985460 (+ 2995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":356,"threads_started":1}
W20260809 12:57:28.985939 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:28.977083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:60875 (local address 127.21.235.126:38679)
0809 12:57:28.977305 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:28.977312 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:28.977836 (+ 524us) server_negotiation.cc:407] Connection header received
0809 12:57:28.978034 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:28.978040 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:28.978130 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:28.978231 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:28.979063 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:28.979846 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:28.981089 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:28.981487 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:28.981621 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:28.985665 (+ 4044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.66:60875: BlockingRecv error: recv got EOF from 127.21.235.66:60875 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 09 12:57:29 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:29 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:29.466856 476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:29.457980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.67:49043)
0809 12:57:29.458697 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:29.458714 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:29.458880 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:29.459476 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:29.459480 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:29.459895 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:29.460068 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:29.460076 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:29.461141 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:29.461144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:29.462063 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:29.462074 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:29.462951 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:29.462955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:29.463070 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:29.463634 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:29.463651 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:29.466604 (+ 2953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":361,"threads_started":1}
W20260809 12:57:29.467022 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:29.458024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:49043 (local address 127.21.235.126:38679)
0809 12:57:29.458214 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:29.458221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:29.458803 (+ 582us) server_negotiation.cc:407] Connection header received
0809 12:57:29.459172 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:29.459175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:29.459259 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:29.459427 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:29.460234 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:29.461017 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:29.462200 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:29.462825 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:29.462923 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:29.466812 (+ 3889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.67:49043: BlockingRecv error: recv got EOF from 127.21.235.67:49043 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 12:57:29 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:29 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:29.532223 477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:29.523492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:41381)
0809 12:57:29.524064 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:29.524081 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:29.524239 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:29.524586 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:29.524591 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:29.525052 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:29.525201 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:29.525210 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:29.526239 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:29.526246 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:29.527245 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:29.527256 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:29.528077 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:29.528083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:29.528249 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:29.528831 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:29.528876 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:29.531988 (+ 3112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":305,"threads_started":1}
W20260809 12:57:29.532416 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:29.523719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:41381 (local address 127.21.235.126:38679)
0809 12:57:29.523937 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:29.523943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:29.524167 (+ 224us) server_negotiation.cc:407] Connection header received
0809 12:57:29.524364 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:29.524369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:29.524444 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:29.524552 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:29.525350 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:29.526096 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:29.527393 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:29.527883 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:29.528019 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:29.532183 (+ 4164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.65:41381: BlockingRecv error: recv got EOF from 127.21.235.65:41381 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 12:57:29 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:29 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:29.996109 478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:29.986926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:55545)
0809 12:57:29.987500 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:29.987518 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:29.987676 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:29.988037 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:29.988044 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:29.988489 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:29.988641 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:29.988649 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:29.989844 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:29.989851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:29.990834 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:29.990846 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:29.991673 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:29.991681 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:29.991847 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:29.992544 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:29.992564 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:29.995876 (+ 3312us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":300,"threads_started":1}
W20260809 12:57:29.996330 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:29.986976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:55545 (local address 127.21.235.126:38679)
0809 12:57:29.987189 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:29.987198 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:29.987612 (+ 414us) server_negotiation.cc:407] Connection header received
0809 12:57:29.987812 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:29.987823 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:29.987885 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:29.987988 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:29.988791 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:29.989695 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:29.990984 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:29.991477 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:29.991612 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:29.996073 (+ 4461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.66:55545: BlockingRecv error: recv got EOF from 127.21.235.66:55545 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 12:57:30 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:30 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:30.476593 479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:30.467892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.67:55811)
0809 12:57:30.468516 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:30.468533 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:30.468648 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:30.468942 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:30.468945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:30.469384 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:30.469543 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:30.469553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:30.470556 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:30.470561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:30.471524 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:30.471538 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:30.472230 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:30.472234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:30.472389 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:30.472999 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:30.473016 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:30.476236 (+ 3220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":327,"threads_started":1}
W20260809 12:57:30.476747 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:30.467969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:55811 (local address 127.21.235.126:38679)
0809 12:57:30.468144 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:30.468149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:30.468607 (+ 458us) server_negotiation.cc:407] Connection header received
0809 12:57:30.468743 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:30.468746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:30.468816 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:30.468901 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:30.469675 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:30.470421 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:30.471650 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:30.472058 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:30.472148 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:30.476441 (+ 4293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.67:55811: BlockingRecv error: recv got EOF from 127.21.235.67:55811 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 12:57:30 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:30 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:30.543242 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:30.533486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:52317)
0809 12:57:30.534064 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:30.534082 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:30.534257 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:30.534617 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:30.534622 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:30.535133 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:30.535383 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:30.535396 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:30.536742 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:30.536745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:30.537688 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:30.537698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:30.538476 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:30.538485 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:30.538732 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:30.539336 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:30.539354 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:30.542828 (+ 3474us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":318,"threads_started":1}
W20260809 12:57:30.543329 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:30.533589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:52317 (local address 127.21.235.126:38679)
0809 12:57:30.533839 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:30.533847 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:30.534174 (+ 327us) server_negotiation.cc:407] Connection header received
0809 12:57:30.534392 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:30.534413 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:30.534474 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:30.534583 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:30.535559 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:30.536628 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:30.537822 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:30.538226 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:30.538365 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:30.543038 (+ 4673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.65:52317: BlockingRecv error: recv got EOF from 127.21.235.65:52317 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 09 12:57:30 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:30 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:31.005390 482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:30.997017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:35691)
0809 12:57:30.997578 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:30.997596 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:30.997713 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:30.998073 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:30.998077 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:30.998564 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:30.998750 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:30.998760 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:30.999804 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:30.999808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:31.000779 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:31.000791 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:31.001482 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:31.001487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:31.001629 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:31.002275 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:31.002293 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:31.005138 (+ 2845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":312,"threads_started":1}
W20260809 12:57:31.005677 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:30.997126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:35691 (local address 127.21.235.126:38679)
0809 12:57:30.997300 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:30.997305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:30.997676 (+ 371us) server_negotiation.cc:407] Connection header received
0809 12:57:30.997838 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:30.997842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:30.997904 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:30.997989 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:30.998890 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:30.999662 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:31.000911 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:31.001318 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:31.001435 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:31.005350 (+ 3915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.66:35691: BlockingRecv error: recv got EOF from 127.21.235.66:35691 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 12:57:31 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:31 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:31.487702 483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:31.477905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.67:33599)
0809 12:57:31.478557 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:31.478577 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:31.478764 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:31.479153 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:31.479158 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:31.479651 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:31.479837 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:31.479847 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:31.481137 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:31.481142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:31.482717 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:31.482732 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:31.483431 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:31.483437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:31.483569 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:31.484159 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:31.484178 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:31.487433 (+ 3255us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":286,"threads_started":1}
W20260809 12:57:31.488121 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:31.477985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:33599 (local address 127.21.235.126:38679)
0809 12:57:31.478236 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:31.478246 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:31.478668 (+ 422us) server_negotiation.cc:407] Connection header received
0809 12:57:31.478904 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:31.478910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:31.479004 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:31.479121 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:31.480000 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:31.480973 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:31.482866 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:31.483275 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:31.483386 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:31.487626 (+ 4240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.67:33599: BlockingRecv error: recv got EOF from 127.21.235.67:33599 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 12:57:31 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:31 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:31.553732 484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:31.544336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:35887)
0809 12:57:31.544970 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:31.544987 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:31.545156 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:31.545543 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:31.545548 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:31.546003 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:31.546167 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:31.546176 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:31.547407 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:31.547415 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:31.548518 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:31.548531 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:31.549404 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:31.549411 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:31.549602 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:31.550204 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:31.550223 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:31.553355 (+ 3132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":399,"threads_started":1}
W20260809 12:57:31.553959 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:31.544448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:35887 (local address 127.21.235.126:38679)
0809 12:57:31.544699 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:31.544709 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:31.545080 (+ 371us) server_negotiation.cc:407] Connection header received
0809 12:57:31.545291 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:31.545296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:31.545384 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:31.545498 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:31.546327 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:31.547245 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:31.548672 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:31.549205 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:31.549375 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:31.553645 (+ 4270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.65:35887: BlockingRecv error: recv got EOF from 127.21.235.65:35887 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 12:57:32 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:32 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:32.015914 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:32.006589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:35681)
0809 12:57:32.007270 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:32.007287 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:32.007402 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:32.007691 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:32.007694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:32.008103 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:32.008246 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:32.008254 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:32.009366 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:32.009371 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:32.010610 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:32.010623 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:32.011635 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:32.011642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:32.011828 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:32.012425 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:32.012444 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:32.015533 (+ 3089us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":380,"threads_started":1}
W20260809 12:57:32.016007 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:32.006610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:35681 (local address 127.21.235.126:38679)
0809 12:57:32.006798 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:32.006803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:32.007377 (+ 574us) server_negotiation.cc:407] Connection header received
0809 12:57:32.007509 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:32.007513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:32.007571 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:32.007648 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:32.008413 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:32.009189 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:32.010826 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:32.011472 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:32.011619 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:32.015760 (+ 4141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.66:35681: BlockingRecv error: recv got EOF from 127.21.235.66:35681 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 12:57:32 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:32 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:32.498653 486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:32.488782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.67:34619)
0809 12:57:32.489464 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:32.489481 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:32.489646 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:32.490038 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:32.490043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:32.490531 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:32.490699 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:32.490709 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:32.491905 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:32.491914 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:32.492971 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:32.492984 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:32.493909 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:32.493916 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:32.494104 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:32.494883 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:32.494904 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:32.498211 (+ 3307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":347,"threads_started":1}
W20260809 12:57:32.498792 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:32.489051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:34619 (local address 127.21.235.126:38679)
0809 12:57:32.489352 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:32.489360 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:32.489579 (+ 219us) server_negotiation.cc:407] Connection header received
0809 12:57:32.489813 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:32.489817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:32.489883 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:32.489996 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:32.490897 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:32.491738 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:32.493127 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:32.493726 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:32.493869 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:32.498506 (+ 4637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.67:34619: BlockingRecv error: recv got EOF from 127.21.235.67:34619 (error 108)
Metrics: {"server-negotiator.queue_time_us":139}
Aug 09 12:57:32 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:32 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:32.564795 487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:32.554940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:52157)
0809 12:57:32.555607 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:32.555633 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:32.555793 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:32.556216 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:32.556220 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:32.556900 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:32.557167 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:32.557181 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:32.558392 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:32.558396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:32.559466 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:32.559479 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:32.560388 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:32.560393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:32.560529 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:32.561178 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:32.561195 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:32.564406 (+ 3211us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":341,"threads_started":1}
W20260809 12:57:32.564896 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:32.555034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:52157 (local address 127.21.235.126:38679)
0809 12:57:32.555294 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:32.555301 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:32.555737 (+ 436us) server_negotiation.cc:407] Connection header received
0809 12:57:32.555933 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:32.555937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:32.556030 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:32.556176 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:32.557341 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:32.558212 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:32.559670 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:32.560183 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:32.560297 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:32.564642 (+ 4345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.65:52157: BlockingRecv error: recv got EOF from 127.21.235.65:52157 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 09 12:57:33 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:33 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:33.025413 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:33.016893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:50007)
0809 12:57:33.017507 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:33.017524 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:33.017641 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:33.017962 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:33.017965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:33.018434 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:33.018692 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:33.018702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:33.019824 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:33.019828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:33.020727 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:33.020738 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:33.021484 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:33.021489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:33.021609 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:33.022200 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:33.022218 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:33.025157 (+ 2939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":343,"threads_started":1}
W20260809 12:57:33.025640 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:33.017031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:50007 (local address 127.21.235.126:38679)
0809 12:57:33.017223 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:33.017230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:33.017608 (+ 378us) server_negotiation.cc:407] Connection header received
0809 12:57:33.017743 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:33.017747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:33.017825 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:33.017916 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:33.018834 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:33.019691 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:33.020882 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:33.021301 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:33.021434 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:33.025404 (+ 3970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.66:50007: BlockingRecv error: recv got EOF from 127.21.235.66:50007 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 12:57:33 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:33 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:33.512914 489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:33.499726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.67:37281)
0809 12:57:33.500244 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:33.500260 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:33.500377 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:33.500899 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:33.500903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:33.501307 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:33.501498 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:33.501507 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:33.502732 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:33.502737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:33.503664 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:33.503676 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:33.504457 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:33.504462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:33.504592 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:33.509155 (+ 4563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:33.509173 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:33.512502 (+ 3329us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":276,"threads_started":1}
W20260809 12:57:33.512960 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:33.499796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:37281 (local address 127.21.235.126:38679)
0809 12:57:33.499989 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:33.499998 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:33.500349 (+ 351us) server_negotiation.cc:407] Connection header received
0809 12:57:33.500496 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:33.500502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:33.500612 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:33.500732 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:33.501648 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:33.502589 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:33.503837 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:33.504298 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:33.504425 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:33.512754 (+ 8329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.67:37281: BlockingRecv error: recv got EOF from 127.21.235.67:37281 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 12:57:33 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:33 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:33.574285 490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:33.565851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:53611)
0809 12:57:33.566366 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:33.566383 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:33.566543 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:33.566857 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:33.566861 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:33.567271 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:33.567445 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:33.567453 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:33.568492 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:33.568496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:33.569436 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:33.569448 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:33.570197 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:33.570201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:33.570330 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:33.571032 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:33.571050 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:33.574044 (+ 2994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":307,"threads_started":1}
W20260809 12:57:33.574450 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:33.565917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:53611 (local address 127.21.235.126:38679)
0809 12:57:33.566193 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:33.566201 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:33.566527 (+ 326us) server_negotiation.cc:407] Connection header received
0809 12:57:33.566647 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:33.566651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:33.566718 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:33.566814 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:33.567576 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:33.568363 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:33.569575 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:33.570075 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:33.570169 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:33.574208 (+ 4039us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.65:53611: BlockingRecv error: recv got EOF from 127.21.235.65:53611 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 09 12:57:34 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:34 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:34.035820 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:34.026572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:43755)
0809 12:57:34.027189 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:34.027237 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:34.027373 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:34.027686 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:34.027689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:34.028098 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:34.028255 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:34.028263 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:34.029485 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:34.029498 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:34.030792 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:34.030805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:34.031560 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:34.031566 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:34.031773 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:34.032439 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:34.032458 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:34.035568 (+ 3110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":376,"threads_started":1}
W20260809 12:57:34.036068 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:34.026595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:43755 (local address 127.21.235.126:38679)
0809 12:57:34.026798 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:34.026803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:34.027343 (+ 540us) server_negotiation.cc:407] Connection header received
0809 12:57:34.027485 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:34.027489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:34.027554 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:34.027645 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:34.028394 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:34.029311 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:34.030940 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:34.031403 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:34.031543 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:34.035814 (+ 4271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.66:43755: BlockingRecv error: recv got EOF from 127.21.235.66:43755 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 12:57:34 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:34 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:34.522768 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:34.513902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.67:51863)
0809 12:57:34.514486 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:34.514504 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:34.514618 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:34.514976 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:34.514980 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:34.515392 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:34.515550 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:34.515559 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:34.516721 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:34.516725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:34.517728 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:34.517740 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:34.518725 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:34.518729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:34.518873 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:34.519492 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:34.519510 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:34.522420 (+ 2910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":323,"threads_started":1}
W20260809 12:57:34.522946 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:34.513979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:51863 (local address 127.21.235.126:38679)
0809 12:57:34.514254 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:34.514262 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:34.514613 (+ 351us) server_negotiation.cc:407] Connection header received
0809 12:57:34.514757 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:34.514761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:34.514832 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:34.514924 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:34.515696 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:34.516579 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:34.517891 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:34.518324 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:34.518488 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:34.522660 (+ 4172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.67:51863: BlockingRecv error: recv got EOF from 127.21.235.67:51863 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 09 12:57:34 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:34 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:34.584645 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:34.575281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:59213)
0809 12:57:34.575850 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:34.575867 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:34.575985 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:34.576402 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:34.576407 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:34.576846 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:34.577029 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:34.577038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:34.578090 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:34.578094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:34.579210 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:34.579224 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:34.580069 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:34.580074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:34.580223 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:34.580866 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:34.580885 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:34.584268 (+ 3383us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":308,"threads_started":1}
W20260809 12:57:34.584820 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:34.575479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:59213 (local address 127.21.235.126:38679)
0809 12:57:34.575678 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:34.575683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:34.575949 (+ 266us) server_negotiation.cc:407] Connection header received
0809 12:57:34.576136 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:34.576140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:34.576205 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:34.576295 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:34.577185 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:34.577944 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:34.579412 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:34.579885 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:34.579982 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:34.584594 (+ 4612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.65:59213: BlockingRecv error: recv got EOF from 127.21.235.65:59213 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 12:57:35 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:35 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:35.046005 494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:35.036815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:41715)
0809 12:57:35.037447 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:35.037464 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:35.037630 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:35.038006 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:35.038011 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:35.038573 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:35.038742 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:35.038751 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:35.039850 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:35.039856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:35.040942 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:35.040955 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:35.041896 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:35.041903 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:35.042103 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:35.042845 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:35.042866 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:35.045749 (+ 2883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":333,"threads_started":1}
W20260809 12:57:35.046255 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:35.037058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:41715 (local address 127.21.235.126:38679)
0809 12:57:35.037322 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:35.037329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:35.037551 (+ 222us) server_negotiation.cc:407] Connection header received
0809 12:57:35.037788 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:35.037793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:35.037858 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:35.037970 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:35.038906 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:35.039694 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:35.041094 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:35.041683 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:35.041846 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:35.045979 (+ 4133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.66:41715: BlockingRecv error: recv got EOF from 127.21.235.66:41715 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 09 12:57:35 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:35 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:35.532711 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:35.523928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.67:36761)
0809 12:57:35.524540 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:35.524557 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:35.524670 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:35.524953 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:35.524955 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:35.525357 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:35.525527 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:35.525536 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:35.526578 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:35.526583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:35.527565 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:35.527578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:35.528460 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:35.528464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:35.528584 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:35.529236 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:35.529253 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:35.532448 (+ 3195us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":306,"threads_started":1}
W20260809 12:57:35.532859 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:35.524011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:36761 (local address 127.21.235.126:38679)
0809 12:57:35.524209 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:35.524214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:35.524644 (+ 430us) server_negotiation.cc:407] Connection header received
0809 12:57:35.524770 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:35.524773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:35.524837 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:35.524937 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:35.525667 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:35.526448 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:35.527715 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:35.528260 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:35.528385 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:35.532661 (+ 4276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.67:36761: BlockingRecv error: recv got EOF from 127.21.235.67:36761 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 09 12:57:35 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:35 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:35.594496 497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:35.585681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:50491)
0809 12:57:35.586199 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:35.586217 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:35.586335 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:35.586644 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:35.586647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:35.587027 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:35.587202 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:35.587209 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:35.588203 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:35.588206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:35.589114 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:35.589125 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:35.589966 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:35.589975 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:35.590152 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:35.590910 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:35.590936 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:35.594107 (+ 3171us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":293,"threads_started":1}
W20260809 12:57:35.594666 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:35.585736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:50491 (local address 127.21.235.126:38679)
0809 12:57:35.586129 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:35.586135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:35.586320 (+ 185us) server_negotiation.cc:407] Connection header received
0809 12:57:35.586460 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:35.586464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:35.586525 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:35.586599 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:35.587332 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:35.588075 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:35.589267 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:35.589790 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:35.589924 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:35.594351 (+ 4427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.65:50491: BlockingRecv error: recv got EOF from 127.21.235.65:50491 (error 108)
Metrics: {"server-negotiator.queue_time_us":210}
Aug 09 12:57:36 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:36 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:36.055554 498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:36.046965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:53073)
0809 12:57:36.047538 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:36.047566 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:36.047677 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:36.047959 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:36.047962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:36.048343 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:36.048521 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:36.048529 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:36.049547 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:36.049552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:36.050696 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:36.050710 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:36.051485 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:36.051490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:36.051627 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:36.052284 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:36.052305 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:36.055290 (+ 2985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":332,"threads_started":1}
W20260809 12:57:36.055745 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:36.047180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:53073 (local address 127.21.235.126:38679)
0809 12:57:36.047354 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:36.047359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:36.047636 (+ 277us) server_negotiation.cc:407] Connection header received
0809 12:57:36.047769 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:36.047773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:36.047833 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:36.047912 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:36.048640 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:36.049375 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:36.050872 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:36.051327 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:36.051432 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:36.055503 (+ 4071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.66:53073: BlockingRecv error: recv got EOF from 127.21.235.66:53073 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 12:57:36 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:36 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.67@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 12:57:36.542675 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:36.533681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:50745 (local address 127.21.235.126:38679)
0809 12:57:36.533856 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:36.533860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:36.534203 (+ 343us) server_negotiation.cc:407] Connection header received
0809 12:57:36.534355 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:36.534358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:36.534476 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:36.534555 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:36.535336 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:36.536085 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:36.537312 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:36.537741 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:36.537853 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:36.542367 (+ 4514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.67:50745: BlockingRecv error: recv got EOF from 127.21.235.67:50745 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 12:57:36.543157 499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:36.533589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.67:50745)
0809 12:57:36.534101 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:36.534128 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:36.534242 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:36.534600 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:36.534603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:36.534996 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:36.535175 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:36.535184 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:36.536219 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:36.536222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:36.537177 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:36.537189 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:36.537936 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:36.537940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:36.538071 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:36.538741 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:36.538759 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:36.542768 (+ 4009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":275,"threads_started":1}
Aug 09 12:57:36 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:36 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.65@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:36.604555 500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:36.595868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.65:58365)
0809 12:57:36.596385 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:36.596402 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:36.596545 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:36.596846 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:36.596850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:36.597263 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:36.597454 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:36.597463 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:36.598612 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:36.598616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:36.599766 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:36.599779 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:36.600624 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:36.600630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:36.600806 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:36.601438 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:36.601456 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:36.604292 (+ 2836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":286,"threads_started":1}
W20260809 12:57:36.604764 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:36.595902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:58365 (local address 127.21.235.126:38679)
0809 12:57:36.596151 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:36.596160 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:36.596525 (+ 365us) server_negotiation.cc:407] Connection header received
0809 12:57:36.596653 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:36.596657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:36.596721 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:36.596806 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:36.597609 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:36.598376 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:36.599907 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:36.600450 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:36.600596 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:36.604538 (+ 3942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.65:58365: BlockingRecv error: recv got EOF from 127.21.235.65:58365 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 12:57:37 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:57:37 dist-test-slave-gp6n krb5kdc[31768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.235.66@KRBTEST.COM for oryx/127.21.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:57:37.065783 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:37.056984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:38679 (local address 127.21.235.66:35511)
0809 12:57:37.057457 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:37.057475 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:37.057625 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:37.057986 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:37.057991 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:37.058550 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:37.058724 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:37.058733 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:37.059762 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:37.059768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:37.060696 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:37.060707 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:37.061474 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:37.061480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:37.061645 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:37.062265 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:37.062283 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:37.065424 (+ 3141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.235.126:38679: Server oryx/127.21.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":238,"threads_started":1}
W20260809 12:57:37.065915 32607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:37.056851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:35511 (local address 127.21.235.126:38679)
0809 12:57:37.057045 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:37.057050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:37.057560 (+ 510us) server_negotiation.cc:407] Connection header received
0809 12:57:37.057759 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:37.057763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:37.057829 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:37.057931 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:37.058869 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:37.059611 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:37.060840 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:37.061314 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:37.061453 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:37.065631 (+ 4178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.235.66:35511: BlockingRecv error: recv got EOF from 127.21.235.66:35511 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 12:57:37.358639 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 32436
I20260809 12:57:37.371515 505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:37.359726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48202 (local address 127.21.235.65:45479)
0809 12:57:37.360402 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:37.360410 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:37.360438 (+ 28us) server_negotiation.cc:407] Connection header received
0809 12:57:37.360500 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:37.360506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:37.360726 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:37.360916 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:37.362538 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:37.363815 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:37.365066 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:37.365665 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:37.365768 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:37.368295 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:37.368304 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:37.368316 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:37.368351 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:37.371008 (+ 2657us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.235.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 12:57:37.371283 (+ 275us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48202: Request ticket server kudu/127.21.235.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":367,"threads_started":1}
W20260809 12:57:37.371641 505 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48202: Request ticket server kudu/127.21.235.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 12:57:37.371748 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 32436: Not authorized: Client connection negotiation failed: client connection to 127.21.235.65:45479: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.235.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 12:57:37.371979 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 32436
I20260809 12:57:37.391492 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 32610
I20260809 12:57:37.403805 506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:37.392458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39282 (local address 127.21.235.66:36933)
0809 12:57:37.393064 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:37.393072 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:37.393103 (+ 31us) server_negotiation.cc:407] Connection header received
0809 12:57:37.393166 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:37.393173 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:37.393402 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:37.393614 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:37.395266 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:37.396347 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:37.397514 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:37.398051 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:37.398146 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:37.400660 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:37.400668 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:37.400682 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:37.400714 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:37.403310 (+ 2596us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.235.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 12:57:37.403570 (+ 260us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39282: Request ticket server kudu/127.21.235.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":273,"threads_started":1}
W20260809 12:57:37.403972 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 32610: Not authorized: Client connection negotiation failed: client connection to 127.21.235.66:36933: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.235.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 12:57:37.404152 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 32610
I20260809 12:57:37.422860 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 32763
I20260809 12:57:37.435364 507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:37.423960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39344 (local address 127.21.235.67:45079)
0809 12:57:37.424599 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:37.424606 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:37.424633 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:57:37.424693 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:37.424699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:37.424928 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:37.425080 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:37.426398 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:37.427471 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:37.428728 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:37.429283 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:37.429379 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:37.432237 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:37.432245 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:37.432258 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:37.432288 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:37.434833 (+ 2545us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.235.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 12:57:37.435082 (+ 249us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39344: Request ticket server kudu/127.21.235.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":345,"threads_started":1}
W20260809 12:57:37.435482 507 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39344: Request ticket server kudu/127.21.235.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 12:57:37.435503 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 32763: Not authorized: Client connection negotiation failed: client connection to 127.21.235.67:45079: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.235.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 12:57:37.435689 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 32763
I20260809 12:57:37.459690 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 32358
I20260809 12:57:37.461050 32429 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:37.580463 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 32358
2026-08-09T12:57:37Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21354 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T12:57:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:57:37Z Disabled control of system clock
I20260809 12:57:37.669035 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:34283
--webserver_interface=127.21.235.126
--webserver_port=0
--builtin_ntp_servers=127.21.235.84:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:34283
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 12:57:38.017793 514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:38.018123 514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:38.018203 514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:38.025524 514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:57:38.025624 514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:38.025666 514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:38.025723 514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:57:38.025768 514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:57:38.034957 514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:34283
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:34283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:38.036214 514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:38.037994 514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:38.050882 521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:38.050910 523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:38.051062 520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:38.051061 514 server_base.cc:1034] running on GCE node
I20260809 12:57:38.052238 514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:38.053943 514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:38.055203 514 hybrid_clock.cc:648] HybridClock initialized: now 1786280258055150 us; error 65 us; skew 500 ppm
I20260809 12:57:38.055591 514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:57:38.058094 514 webserver.cc:460] Webserver started at http://127.21.235.126:42879/ using document root <none> and password file <none>
I20260809 12:57:38.058924 514 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:38.059057 514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:38.059419 514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:38.062618 514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "4a52efc961c540bea07861ca0d01cdc7"
format_stamp: "Formatted at 2026-08-09 12:57:38 on dist-test-slave-gp6n"
server_key: "968601c49116f3b5810b3bc9b171ab53"
server_key_iv: "e4e8fc8677672901faf33eb50ed9000a"
server_key_version: "encryptionkey@0"
I20260809 12:57:38.063469 514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "4a52efc961c540bea07861ca0d01cdc7"
format_stamp: "Formatted at 2026-08-09 12:57:38 on dist-test-slave-gp6n"
server_key: "968601c49116f3b5810b3bc9b171ab53"
server_key_iv: "e4e8fc8677672901faf33eb50ed9000a"
server_key_version: "encryptionkey@0"
I20260809 12:57:38.068657 514 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:57:38.072695 529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:38.074097 514 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:57:38.074283 514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "4a52efc961c540bea07861ca0d01cdc7"
format_stamp: "Formatted at 2026-08-09 12:57:38 on dist-test-slave-gp6n"
server_key: "968601c49116f3b5810b3bc9b171ab53"
server_key_iv: "e4e8fc8677672901faf33eb50ed9000a"
server_key_version: "encryptionkey@0"
I20260809 12:57:38.074529 514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:38.094521 514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:38.095408 514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:38.095703 514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:38.115974 514 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:34283
I20260809 12:57:38.115993 583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:34283 every 8 connection(s)
I20260809 12:57:38.117355 514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:57:38.121297 584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:38.125233 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 514
I20260809 12:57:38.125509 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:57:38.126255 22445 external_mini_cluster.cc:1428] Setting key bcac2beebb3cd99fab2111e39b5b8179
I20260809 12:57:38.132586 584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7: Bootstrap starting.
I20260809 12:57:38.135775 584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:38.136830 584 log.cc:824] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7: Log is configured to *not* fsync() on all Append() calls
I20260809 12:57:38.140228 584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7: No bootstrap required, opened a new log
I20260809 12:57:38.141124 587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:38.128464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33412 (local address 127.21.235.126:34283)
0809 12:57:38.129650 (+ 1186us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:38.129662 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:38.129705 (+ 43us) server_negotiation.cc:407] Connection header received
0809 12:57:38.131718 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:38.131741 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:38.132225 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:38.132629 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:38.133278 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:38.134873 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:38.136430 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:38.137269 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:38.137499 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:38.138801 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:38.138808 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:38.138822 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:57:38.139375 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:38.139452 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:57:38.139473 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:38.139569 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:38.139829 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 12:57:38.140138 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":381,"threads_started":1}
I20260809 12:57:38.149464 584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a52efc961c540bea07861ca0d01cdc7" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34283 } }
I20260809 12:57:38.150040 584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:38.150177 584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a52efc961c540bea07861ca0d01cdc7, State: Initialized, Role: FOLLOWER
I20260809 12:57:38.150787 584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a52efc961c540bea07861ca0d01cdc7" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34283 } }
I20260809 12:57:38.151005 584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:57:38.151109 584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:57:38.151242 584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:38.153553 584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a52efc961c540bea07861ca0d01cdc7" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34283 } }
I20260809 12:57:38.154038 584 leader_election.cc:302] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a52efc961c540bea07861ca0d01cdc7; no voters:
I20260809 12:57:38.154780 584 leader_election.cc:288] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:57:38.154891 590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:57:38.156368 590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7 [term 1 LEADER]: Becoming Leader. State: Replica: 4a52efc961c540bea07861ca0d01cdc7, State: Running, Role: LEADER
I20260809 12:57:38.156929 590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a52efc961c540bea07861ca0d01cdc7" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34283 } }
I20260809 12:57:38.157321 584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:57:38.159822 591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a52efc961c540bea07861ca0d01cdc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a52efc961c540bea07861ca0d01cdc7" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34283 } } }
I20260809 12:57:38.160327 591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7 [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:38.161012 593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a52efc961c540bea07861ca0d01cdc7. Latest consensus state: current_term: 1 leader_uuid: "4a52efc961c540bea07861ca0d01cdc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a52efc961c540bea07861ca0d01cdc7" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34283 } } }
I20260809 12:57:38.162541 593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7 [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:38.164327 600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:57:38.169828 600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:57:38.179710 600 catalog_manager.cc:1347] Generated new cluster ID: fd995ef443ff4a16833ef3d398c98014
I20260809 12:57:38.179839 600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:57:38.202142 600 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:57:38.203660 600 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:57:38.211793 600 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a52efc961c540bea07861ca0d01cdc7: Generated new TSK 0
I20260809 12:57:38.212980 600 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:57:38.228833 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:34283
--builtin_ntp_servers=127.21.235.84:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 12:57:38.575438 612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:38.575752 612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:38.575911 612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:38.583392 612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:38.583524 612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:38.583707 612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:57:38.592885 612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:34283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:38.594197 612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:38.595992 612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:38.609195 618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:38.609225 621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:38.609256 619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:38.610498 612 server_base.cc:1034] running on GCE node
I20260809 12:57:38.611094 612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:38.611975 612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:38.613175 612 hybrid_clock.cc:648] HybridClock initialized: now 1786280258613115 us; error 57 us; skew 500 ppm
I20260809 12:57:38.613513 612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:57:38.621140 612 webserver.cc:460] Webserver started at http://127.21.235.65:41203/ using document root <none> and password file <none>
I20260809 12:57:38.621856 612 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:38.621948 612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:38.622223 612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:38.625178 612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "3be90b3690ce498fafa1ba84f280d114"
format_stamp: "Formatted at 2026-08-09 12:57:38 on dist-test-slave-gp6n"
server_key: "7b7c6d25668ad59cc496e1a1f0d0a9fa"
server_key_iv: "247ac86b1f2524bb38e73feea2b7ee74"
server_key_version: "encryptionkey@0"
I20260809 12:57:38.625931 612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "3be90b3690ce498fafa1ba84f280d114"
format_stamp: "Formatted at 2026-08-09 12:57:38 on dist-test-slave-gp6n"
server_key: "7b7c6d25668ad59cc496e1a1f0d0a9fa"
server_key_iv: "247ac86b1f2524bb38e73feea2b7ee74"
server_key_version: "encryptionkey@0"
I20260809 12:57:38.631163 612 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:57:38.634670 627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:38.635957 612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:57:38.636096 612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "3be90b3690ce498fafa1ba84f280d114"
format_stamp: "Formatted at 2026-08-09 12:57:38 on dist-test-slave-gp6n"
server_key: "7b7c6d25668ad59cc496e1a1f0d0a9fa"
server_key_iv: "247ac86b1f2524bb38e73feea2b7ee74"
server_key_version: "encryptionkey@0"
I20260809 12:57:38.636273 612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:38.651664 612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:38.652472 612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:38.652736 612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:38.653630 612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:38.655140 612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:38.655230 612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:38.655313 612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:38.655371 612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:38.689184 612 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:41377
I20260809 12:57:38.689287 755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:41377 every 8 connection(s)
I20260809 12:57:38.690605 612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:57:38.699215 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 612
I20260809 12:57:38.699412 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:57:38.700032 22445 external_mini_cluster.cc:1428] Setting key 5156470f4ca0ffb6eebccb8bdafa83d0
I20260809 12:57:38.702701 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:34283
--builtin_ntp_servers=127.21.235.84:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 12:57:38.703681 758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:38.693229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:41803 (local address 127.21.235.126:34283)
0809 12:57:38.693702 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:38.693706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:38.694869 (+ 1163us) server_negotiation.cc:407] Connection header received
0809 12:57:38.696795 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:38.696801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:38.696898 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:38.697016 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:38.698066 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:38.698906 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:38.700575 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:38.701105 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:38.701244 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:38.702789 (+ 1545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:38.702795 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:38.702800 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:57:38.702821 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:38.702881 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:57:38.702899 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:38.702988 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:38.703209 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 12:57:38.703320 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":368,"thread_start_us":262,"threads_started":1}
I20260809 12:57:38.704654 759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:38.693757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:34283 (local address 127.21.235.65:41803)
0809 12:57:38.694707 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:38.694742 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:38.696525 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:38.697250 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:38.697258 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:38.697346 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:38.697899 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:38.697916 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:38.699069 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:38.699073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:38.700436 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:38.700448 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:38.701332 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:38.701340 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:38.701675 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:38.702491 (+ 816us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:57:38.702494 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:38.702534 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:57:38.702606 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:38.702987 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:38.702999 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:38.703009 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 12:57:38.703198 (+ 189us) client_negotiation.cc:240] Negotiation successful
0809 12:57:38.703581 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":272,"threads_started":1}
I20260809 12:57:38.706260 756 heartbeater.cc:347] Connected to a master server at 127.21.235.126:34283
I20260809 12:57:38.706580 756 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:38.707338 756 heartbeater.cc:511] Master 127.21.235.126:34283 requested a full tablet report, sending...
I20260809 12:57:38.709647 547 ts_manager.cc:194] Registered new tserver with Master: 3be90b3690ce498fafa1ba84f280d114 (127.21.235.65:41377)
I20260809 12:57:38.711918 547 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.235.65:41803
W20260809 12:57:39.043860 760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:39.044149 760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:39.044235 760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:39.051913 760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:39.052017 760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:39.052124 760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:57:39.061640 760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:34283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:39.063074 760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:39.064807 760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:39.079202 768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:39.079233 766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:39.079466 765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:39.080089 760 server_base.cc:1034] running on GCE node
I20260809 12:57:39.080755 760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:39.081681 760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:39.082882 760 hybrid_clock.cc:648] HybridClock initialized: now 1786280259082823 us; error 58 us; skew 500 ppm
I20260809 12:57:39.083209 760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:57:39.085580 760 webserver.cc:460] Webserver started at http://127.21.235.66:38055/ using document root <none> and password file <none>
I20260809 12:57:39.086299 760 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:39.086447 760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:39.086782 760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:39.089656 760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "d0291d04b9024cf289d6c4554c3772f8"
format_stamp: "Formatted at 2026-08-09 12:57:39 on dist-test-slave-gp6n"
server_key: "4cfe46e9b09471bc4aca9a2143342d6a"
server_key_iv: "fada0a7060823a39e698b1c400a0939a"
server_key_version: "encryptionkey@0"
I20260809 12:57:39.090587 760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "d0291d04b9024cf289d6c4554c3772f8"
format_stamp: "Formatted at 2026-08-09 12:57:39 on dist-test-slave-gp6n"
server_key: "4cfe46e9b09471bc4aca9a2143342d6a"
server_key_iv: "fada0a7060823a39e698b1c400a0939a"
server_key_version: "encryptionkey@0"
I20260809 12:57:39.096248 760 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:57:39.100171 775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:39.101707 760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:57:39.101928 760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "d0291d04b9024cf289d6c4554c3772f8"
format_stamp: "Formatted at 2026-08-09 12:57:39 on dist-test-slave-gp6n"
server_key: "4cfe46e9b09471bc4aca9a2143342d6a"
server_key_iv: "fada0a7060823a39e698b1c400a0939a"
server_key_version: "encryptionkey@0"
I20260809 12:57:39.102183 760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:39.123180 760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:39.124162 760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:39.124508 760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:39.125624 760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:39.127311 760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:39.127455 760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:39.127604 760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:39.127703 760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:39.163528 760 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:38679
I20260809 12:57:39.163576 899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:38679 every 8 connection(s)
I20260809 12:57:39.164902 760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:57:39.172183 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 760
I20260809 12:57:39.172354 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:57:39.172971 22445 external_mini_cluster.cc:1428] Setting key 66d46cc39abe5b9660e0b00b691e0740
I20260809 12:57:39.175937 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:34283
--builtin_ntp_servers=127.21.235.84:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 12:57:39.178371 758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:39.167649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:38955 (local address 127.21.235.126:34283)
0809 12:57:39.167872 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:39.167879 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:39.169386 (+ 1507us) server_negotiation.cc:407] Connection header received
0809 12:57:39.171233 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:39.171237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:39.171306 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:39.171398 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:39.172568 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:39.173637 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:39.175585 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:39.175977 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:39.176106 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:39.177541 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:39.177546 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:39.177549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:57:39.177566 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:39.177597 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:57:39.177609 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:39.177703 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:39.177965 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 12:57:39.178061 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:57:39.179452 902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:39.168287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:34283 (local address 127.21.235.66:38955)
0809 12:57:39.169199 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:39.169236 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:39.170952 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:39.171629 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:39.171636 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:39.171724 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:39.172371 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:39.172391 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:39.173791 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:39.173796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:39.175417 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:39.175435 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:39.176207 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:39.176215 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:39.176442 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:39.177184 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:57:39.177187 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:39.177226 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:57:39.177298 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:39.177703 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:39.177715 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:39.177725 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 12:57:39.177965 (+ 240us) client_negotiation.cc:240] Negotiation successful
0809 12:57:39.178391 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":287,"threads_started":1}
I20260809 12:57:39.180783 900 heartbeater.cc:347] Connected to a master server at 127.21.235.126:34283
I20260809 12:57:39.181088 900 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:39.181802 900 heartbeater.cc:511] Master 127.21.235.126:34283 requested a full tablet report, sending...
I20260809 12:57:39.183218 547 ts_manager.cc:194] Registered new tserver with Master: d0291d04b9024cf289d6c4554c3772f8 (127.21.235.66:38679)
I20260809 12:57:39.184432 547 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.235.66:38955
W20260809 12:57:39.516070 903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:39.516342 903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:39.516424 903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:39.523554 903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:39.523651 903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:39.523753 903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:57:39.532625 903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:34283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:39.533783 903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:39.535512 903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:39.548341 913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:39.548411 910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:39.548507 909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:39.549557 903 server_base.cc:1034] running on GCE node
I20260809 12:57:39.550134 903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:39.551126 903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:39.552322 903 hybrid_clock.cc:648] HybridClock initialized: now 1786280259552268 us; error 41 us; skew 500 ppm
I20260809 12:57:39.552659 903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:57:39.554931 903 webserver.cc:460] Webserver started at http://127.21.235.67:33583/ using document root <none> and password file <none>
I20260809 12:57:39.555619 903 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:39.555734 903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:39.556070 903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:39.559036 903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "cad61f494c834f9fb46192d8b5b5ecd2"
format_stamp: "Formatted at 2026-08-09 12:57:39 on dist-test-slave-gp6n"
server_key: "690c066fb02ab7f2e5d4f936e58a3e94"
server_key_iv: "93a3e37935fc7725ce5a6873aec0c16a"
server_key_version: "encryptionkey@0"
I20260809 12:57:39.559846 903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "cad61f494c834f9fb46192d8b5b5ecd2"
format_stamp: "Formatted at 2026-08-09 12:57:39 on dist-test-slave-gp6n"
server_key: "690c066fb02ab7f2e5d4f936e58a3e94"
server_key_iv: "93a3e37935fc7725ce5a6873aec0c16a"
server_key_version: "encryptionkey@0"
I20260809 12:57:39.565100 903 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:57:39.568482 920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:39.569801 903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:57:39.569981 903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "cad61f494c834f9fb46192d8b5b5ecd2"
format_stamp: "Formatted at 2026-08-09 12:57:39 on dist-test-slave-gp6n"
server_key: "690c066fb02ab7f2e5d4f936e58a3e94"
server_key_iv: "93a3e37935fc7725ce5a6873aec0c16a"
server_key_version: "encryptionkey@0"
I20260809 12:57:39.570194 903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:39.583364 903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:39.584180 903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:39.584501 903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:39.585445 903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:39.586925 903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:39.587049 903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:39.587188 903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:39.587278 903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:39.621630 903 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:43251
I20260809 12:57:39.621719 1050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:43251 every 8 connection(s)
I20260809 12:57:39.623016 903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:57:39.633409 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 903
I20260809 12:57:39.633622 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:57:39.634181 22445 external_mini_cluster.cc:1428] Setting key 43262c459a009dd8cffed31ccfa014be
I20260809 12:57:39.636592 758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:39.625736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:52525 (local address 127.21.235.126:34283)
0809 12:57:39.625954 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:39.625959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:39.627556 (+ 1597us) server_negotiation.cc:407] Connection header received
0809 12:57:39.629485 (+ 1929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:39.629493 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:39.629570 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:39.629661 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:39.630832 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:39.631575 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:39.633455 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:39.633969 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:39.634065 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:39.635711 (+ 1646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:39.635716 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:39.635719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:57:39.635742 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:39.635775 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:57:39.635786 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:39.635856 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:39.636167 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 12:57:39.636278 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 12:57:39.637841 1053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:39.626254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:34283 (local address 127.21.235.67:52525)
0809 12:57:39.627400 (+ 1146us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:39.627438 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:39.629148 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:39.629896 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:39.629905 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:39.629992 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:39.630674 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:39.630693 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:39.631729 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:39.631733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:39.633301 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:39.633319 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:39.634202 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:39.634213 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:39.634568 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:39.635413 (+ 845us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:57:39.635415 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:39.635455 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:57:39.635530 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:39.635891 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:39.635904 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:39.635914 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 12:57:39.636126 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 12:57:39.636536 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":463,"threads_started":1}
I20260809 12:57:39.639248 1051 heartbeater.cc:347] Connected to a master server at 127.21.235.126:34283
I20260809 12:57:39.639611 1051 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:39.640477 1051 heartbeater.cc:511] Master 127.21.235.126:34283 requested a full tablet report, sending...
I20260809 12:57:39.641839 547 ts_manager.cc:194] Registered new tserver with Master: cad61f494c834f9fb46192d8b5b5ecd2 (127.21.235.67:43251)
I20260809 12:57:39.642745 547 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.235.67:52525
I20260809 12:57:39.649765 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 12:57:39.656359 758 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:39.654728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33422 (local address 127.21.235.126:34283)
0809 12:57:39.654987 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:39.654997 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:39.655434 (+ 437us) server_negotiation.cc:407] Connection header received
0809 12:57:39.655569 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:39.655573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:39.655646 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:39.655725 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:39.656149 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33422: BlockingRecv error: recv got EOF from 127.0.0.1:33422 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:57:39.656827 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 612
I20260809 12:57:39.665364 1061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:39.657856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60876 (local address 127.21.235.65:41377)
0809 12:57:39.658475 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:39.658481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:39.658507 (+ 26us) server_negotiation.cc:407] Connection header received
0809 12:57:39.658569 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:39.658575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:39.658811 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:39.658947 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:39.659301 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:39.660427 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:39.661913 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:39.662608 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:39.662708 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:39.663752 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:39.663760 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:39.663772 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:57:39.664476 (+ 704us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:39.664529 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:57:39.664549 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:39.664640 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:39.664831 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 12:57:39.664980 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":295,"threads_started":1}
I20260809 12:57:39.666620 750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:39.747893 756 heartbeater.cc:503] Master 127.21.235.126:34283 was elected leader, sending a full tablet report...
I20260809 12:57:39.749310 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 612
W20260809 12:57:39.767715 533 connection.cc:535] server connection from 127.21.235.65:41803 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 12:57:39.768039 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 760
I20260809 12:57:39.775998 1063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:39.768921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60004 (local address 127.21.235.66:38679)
0809 12:57:39.769479 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:39.769487 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:39.769515 (+ 28us) server_negotiation.cc:407] Connection header received
0809 12:57:39.769586 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:39.769592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:39.769815 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:39.769947 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:39.770327 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:39.771423 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:39.772641 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:39.773210 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:39.773315 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:39.774358 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:39.774367 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:39.774381 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:57:39.775202 (+ 821us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:39.775254 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:57:39.775279 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:39.775390 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:39.775505 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 12:57:39.775639 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":277,"threads_started":1}
I20260809 12:57:39.777442 894 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:39.859129 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 760
I20260809 12:57:39.876935 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 903
I20260809 12:57:39.885104 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:39.877981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56908 (local address 127.21.235.67:43251)
0809 12:57:39.878727 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:39.878742 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:39.878773 (+ 31us) server_negotiation.cc:407] Connection header received
0809 12:57:39.878895 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:39.878902 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:39.879132 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:39.879293 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:39.879688 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:39.880741 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:39.881965 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:39.882582 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:39.882679 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:39.883712 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:39.883720 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:39.883732 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:57:39.884316 (+ 584us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:39.884361 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:57:39.884385 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:39.884475 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:39.884652 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 12:57:39.884770 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":439,"threads_started":1}
I20260809 12:57:39.886333 1045 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:39.969897 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 903
I20260809 12:57:39.988236 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 514
I20260809 12:57:39.989662 579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:40.094703 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 514
2026-08-09T12:57:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2509 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T12:57:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:57:40Z Disabled control of system clock
I20260809 12:57:40.179759 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:43115
--webserver_interface=127.21.235.126
--webserver_port=0
--builtin_ntp_servers=127.21.235.84:36345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:43115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 12:57:40.526986 1071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:40.527298 1071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:40.527391 1071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:40.534659 1071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:57:40.534765 1071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:40.534803 1071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:40.534875 1071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:57:40.534921 1071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:57:40.544095 1071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:36345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:43115
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:43115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:40.545372 1071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:40.547163 1071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:40.559513 1081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:40.559634 1077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:40.559597 1079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:40.560577 1071 server_base.cc:1034] running on GCE node
I20260809 12:57:40.561424 1071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:40.562994 1071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:40.564229 1071 hybrid_clock.cc:648] HybridClock initialized: now 1786280260564176 us; error 44 us; skew 500 ppm
I20260809 12:57:40.564580 1071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:57:40.567025 1071 webserver.cc:460] Webserver started at http://127.21.235.126:46593/ using document root <none> and password file <none>
I20260809 12:57:40.567718 1071 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:40.567858 1071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:40.568190 1071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:40.572886 1071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "f798d6f82e4c480a97cba3b0ea70fac2"
format_stamp: "Formatted at 2026-08-09 12:57:40 on dist-test-slave-gp6n"
server_key: "f49a089ae7f337c23868d09a13ef5833"
server_key_iv: "d5469037de94f80965034d84c309a0ef"
server_key_version: "encryptionkey@0"
I20260809 12:57:40.573763 1071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "f798d6f82e4c480a97cba3b0ea70fac2"
format_stamp: "Formatted at 2026-08-09 12:57:40 on dist-test-slave-gp6n"
server_key: "f49a089ae7f337c23868d09a13ef5833"
server_key_iv: "d5469037de94f80965034d84c309a0ef"
server_key_version: "encryptionkey@0"
I20260809 12:57:40.579178 1071 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 12:57:40.582790 1088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:40.584156 1071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:57:40.584350 1071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "f798d6f82e4c480a97cba3b0ea70fac2"
format_stamp: "Formatted at 2026-08-09 12:57:40 on dist-test-slave-gp6n"
server_key: "f49a089ae7f337c23868d09a13ef5833"
server_key_iv: "d5469037de94f80965034d84c309a0ef"
server_key_version: "encryptionkey@0"
I20260809 12:57:40.584591 1071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:40.590600 1071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:40.591368 1071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:40.591674 1071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:40.610860 1071 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:43115
I20260809 12:57:40.610898 1146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:43115 every 8 connection(s)
I20260809 12:57:40.612244 1071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:57:40.614193 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 1071
I20260809 12:57:40.614516 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:57:40.615162 22445 external_mini_cluster.cc:1428] Setting key deb022b0cdd91de81242fab039c57219
I20260809 12:57:40.616266 1147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:40.624981 1150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:40.617223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40424 (local address 127.21.235.126:43115)
0809 12:57:40.618340 (+ 1117us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:40.618354 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:40.618430 (+ 76us) server_negotiation.cc:407] Connection header received
0809 12:57:40.620915 (+ 2485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:40.620944 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:40.621506 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:40.621947 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:40.623014 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:40.623028 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:40.623046 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:57:40.623067 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:40.623142 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:57:40.623161 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:40.623401 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:40.623531 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 12:57:40.623803 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":369,"threads_started":1}
I20260809 12:57:40.630316 1147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2: Bootstrap starting.
I20260809 12:57:40.633384 1147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:40.634308 1147 log.cc:824] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2: Log is configured to *not* fsync() on all Append() calls
I20260809 12:57:40.637082 1147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2: No bootstrap required, opened a new log
I20260809 12:57:40.644167 1147 raft_consensus.cc:348] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f798d6f82e4c480a97cba3b0ea70fac2" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43115 } }
I20260809 12:57:40.644542 1147 raft_consensus.cc:374] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:40.644657 1147 raft_consensus.cc:729] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f798d6f82e4c480a97cba3b0ea70fac2, State: Initialized, Role: FOLLOWER
I20260809 12:57:40.645156 1147 consensus_queue.cc:260] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f798d6f82e4c480a97cba3b0ea70fac2" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43115 } }
I20260809 12:57:40.645360 1147 raft_consensus.cc:388] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:57:40.645455 1147 raft_consensus.cc:482] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:57:40.645573 1147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:40.647614 1147 raft_consensus.cc:504] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f798d6f82e4c480a97cba3b0ea70fac2" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43115 } }
I20260809 12:57:40.648030 1147 leader_election.cc:302] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f798d6f82e4c480a97cba3b0ea70fac2; no voters:
I20260809 12:57:40.648572 1147 leader_election.cc:288] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:57:40.648734 1152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:57:40.649475 1152 raft_consensus.cc:686] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2 [term 1 LEADER]: Becoming Leader. State: Replica: f798d6f82e4c480a97cba3b0ea70fac2, State: Running, Role: LEADER
I20260809 12:57:40.649988 1152 consensus_queue.cc:237] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f798d6f82e4c480a97cba3b0ea70fac2" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43115 } }
I20260809 12:57:40.650717 1147 sys_catalog.cc:564] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:57:40.653533 1153 sys_catalog.cc:455] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f798d6f82e4c480a97cba3b0ea70fac2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f798d6f82e4c480a97cba3b0ea70fac2" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43115 } } }
I20260809 12:57:40.653505 1154 sys_catalog.cc:455] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f798d6f82e4c480a97cba3b0ea70fac2. Latest consensus state: current_term: 1 leader_uuid: "f798d6f82e4c480a97cba3b0ea70fac2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f798d6f82e4c480a97cba3b0ea70fac2" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43115 } } }
I20260809 12:57:40.654048 1153 sys_catalog.cc:458] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2 [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:40.654102 1154 sys_catalog.cc:458] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2 [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:40.657950 1160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:57:40.663621 1160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:57:40.673013 1160 catalog_manager.cc:1347] Generated new cluster ID: 72543b1c53ae4e4bae83bd4df0a9e179
I20260809 12:57:40.673280 1160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:57:40.688788 1160 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:57:40.689379 1160 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:57:40.701104 1160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f798d6f82e4c480a97cba3b0ea70fac2: Generated new TSK 0
I20260809 12:57:40.702210 1160 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:57:40.721160 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:43115
--builtin_ntp_servers=127.21.235.84:36345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 12:57:41.059013 1174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:41.059357 1174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:41.059527 1174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:41.066847 1174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:41.066988 1174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:41.067162 1174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:57:41.076165 1174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:36345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:43115
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:41.077441 1174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:41.079332 1174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:41.092173 1181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:41.092216 1180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:41.092269 1183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:41.093189 1174 server_base.cc:1034] running on GCE node
I20260809 12:57:41.093787 1174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:41.094794 1174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:41.096031 1174 hybrid_clock.cc:648] HybridClock initialized: now 1786280261095993 us; error 54 us; skew 500 ppm
I20260809 12:57:41.096349 1174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:57:41.098594 1174 webserver.cc:460] Webserver started at http://127.21.235.65:45887/ using document root <none> and password file <none>
I20260809 12:57:41.099277 1174 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:41.099414 1174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:41.099769 1174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:41.102903 1174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "36907cfe26f34c2aaf1ffb7e76c64b0f"
format_stamp: "Formatted at 2026-08-09 12:57:41 on dist-test-slave-gp6n"
server_key: "c1eb16d9805e46ec0e4254c0c88b8897"
server_key_iv: "6912eaafe85f93dc1d98e8a10e193f80"
server_key_version: "encryptionkey@0"
I20260809 12:57:41.103721 1174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "36907cfe26f34c2aaf1ffb7e76c64b0f"
format_stamp: "Formatted at 2026-08-09 12:57:41 on dist-test-slave-gp6n"
server_key: "c1eb16d9805e46ec0e4254c0c88b8897"
server_key_iv: "6912eaafe85f93dc1d98e8a10e193f80"
server_key_version: "encryptionkey@0"
I20260809 12:57:41.108732 1174 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:57:41.112310 1190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:41.113617 1174 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:57:41.113798 1174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "36907cfe26f34c2aaf1ffb7e76c64b0f"
format_stamp: "Formatted at 2026-08-09 12:57:41 on dist-test-slave-gp6n"
server_key: "c1eb16d9805e46ec0e4254c0c88b8897"
server_key_iv: "6912eaafe85f93dc1d98e8a10e193f80"
server_key_version: "encryptionkey@0"
I20260809 12:57:41.114027 1174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:41.119688 1174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:41.120465 1174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:41.120754 1174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:41.121671 1174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:41.123148 1174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:41.123270 1174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:41.123406 1174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:41.123508 1174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:41.157231 1174 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:34429
I20260809 12:57:41.157294 1315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:34429 every 8 connection(s)
I20260809 12:57:41.158737 1174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:57:41.166998 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:41.161329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:36213 (local address 127.21.235.126:43115)
0809 12:57:41.161776 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:41.161781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:41.162886 (+ 1105us) server_negotiation.cc:407] Connection header received
0809 12:57:41.164594 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:41.164600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:41.164676 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:41.164811 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:41.166132 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:41.166138 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:41.166143 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:57:41.166164 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:41.166215 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:57:41.166226 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:41.166379 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:41.166616 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 12:57:41.166689 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":250,"threads_started":1}
I20260809 12:57:41.168145 1319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:41.161783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:43115 (local address 127.21.235.65:36213)
0809 12:57:41.162730 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:41.162775 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:41.164319 (+ 1544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:41.165046 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:41.165054 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:41.165144 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:41.165835 (+ 691us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:57:41.165837 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:41.165874 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:57:41.165960 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:41.166325 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:41.166336 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:41.166344 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 12:57:41.166685 (+ 341us) client_negotiation.cc:240] Negotiation successful
0809 12:57:41.166928 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":270,"threads_started":1}
I20260809 12:57:41.168411 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 1174
I20260809 12:57:41.168649 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:57:41.169448 22445 external_mini_cluster.cc:1428] Setting key ebc13cf3aa746cc624687eeae2a1a2bd
I20260809 12:57:41.169828 1316 heartbeater.cc:347] Connected to a master server at 127.21.235.126:43115
I20260809 12:57:41.170159 1316 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:41.171026 1316 heartbeater.cc:511] Master 127.21.235.126:43115 requested a full tablet report, sending...
I20260809 12:57:41.172591 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:43115
--builtin_ntp_servers=127.21.235.84:36345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 12:57:41.173175 1106 ts_manager.cc:194] Registered new tserver with Master: 36907cfe26f34c2aaf1ffb7e76c64b0f (127.21.235.65:34429)
W20260809 12:57:41.530983 1320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:41.531281 1320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:41.531481 1320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:41.538859 1320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:41.538951 1320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:41.539119 1320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:57:41.549064 1320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:36345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:43115
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:41.550444 1320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:41.552379 1320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:41.565636 1325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:41.565717 1326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:41.565668 1328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:41.566679 1320 server_base.cc:1034] running on GCE node
I20260809 12:57:41.567358 1320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:41.568303 1320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:41.569514 1320 hybrid_clock.cc:648] HybridClock initialized: now 1786280261569456 us; error 49 us; skew 500 ppm
I20260809 12:57:41.569846 1320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:57:41.572072 1320 webserver.cc:460] Webserver started at http://127.21.235.66:34425/ using document root <none> and password file <none>
I20260809 12:57:41.572722 1320 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:41.572809 1320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:41.573078 1320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:41.575987 1320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "f520f40f6f4b43898b39939e1221e70a"
format_stamp: "Formatted at 2026-08-09 12:57:41 on dist-test-slave-gp6n"
server_key: "266b1f721280e176f34af0ef67843224"
server_key_iv: "0cc3484408c82397d0a332736205f699"
server_key_version: "encryptionkey@0"
I20260809 12:57:41.576749 1320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "f520f40f6f4b43898b39939e1221e70a"
format_stamp: "Formatted at 2026-08-09 12:57:41 on dist-test-slave-gp6n"
server_key: "266b1f721280e176f34af0ef67843224"
server_key_iv: "0cc3484408c82397d0a332736205f699"
server_key_version: "encryptionkey@0"
I20260809 12:57:41.581961 1320 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 12:57:41.585381 1334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:41.586787 1320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 12:57:41.586913 1320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "f520f40f6f4b43898b39939e1221e70a"
format_stamp: "Formatted at 2026-08-09 12:57:41 on dist-test-slave-gp6n"
server_key: "266b1f721280e176f34af0ef67843224"
server_key_iv: "0cc3484408c82397d0a332736205f699"
server_key_version: "encryptionkey@0"
I20260809 12:57:41.587095 1320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:41.592717 1320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:41.593437 1320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:41.593662 1320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:41.594578 1320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:41.595903 1320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:41.595993 1320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:41.596073 1320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:41.596128 1320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:41.636440 1320 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:34957
I20260809 12:57:41.636555 1465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:34957 every 8 connection(s)
I20260809 12:57:41.637765 1320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:57:41.640151 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 1320
I20260809 12:57:41.640385 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:57:41.640908 22445 external_mini_cluster.cc:1428] Setting key 0c41355838aacb5cd960dac54dae180e
I20260809 12:57:41.643653 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:43115
--builtin_ntp_servers=127.21.235.84:36345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 12:57:41.646253 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:41.640368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:41771 (local address 127.21.235.126:43115)
0809 12:57:41.640600 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:41.640606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:41.642186 (+ 1580us) server_negotiation.cc:407] Connection header received
0809 12:57:41.643952 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:41.643960 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:41.644025 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:41.644138 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:41.645436 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:41.645445 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:41.645453 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:57:41.645476 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:41.645522 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:57:41.645539 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:41.645661 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:41.645828 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 12:57:41.645894 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 12:57:41.647055 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:41.640968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:43115 (local address 127.21.235.66:41771)
0809 12:57:41.642025 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:41.642070 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:41.643639 (+ 1569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:41.644324 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:41.644332 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:41.644417 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:41.645140 (+ 723us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:57:41.645142 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:41.645178 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:57:41.645259 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:41.645639 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:41.645647 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:41.645652 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 12:57:41.645854 (+ 202us) client_negotiation.cc:240] Negotiation successful
0809 12:57:41.646038 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":326,"threads_started":1}
I20260809 12:57:41.648465 1466 heartbeater.cc:347] Connected to a master server at 127.21.235.126:43115
I20260809 12:57:41.648743 1466 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:41.649322 1466 heartbeater.cc:511] Master 127.21.235.126:43115 requested a full tablet report, sending...
I20260809 12:57:41.650768 1106 ts_manager.cc:194] Registered new tserver with Master: f520f40f6f4b43898b39939e1221e70a (127.21.235.66:34957)
W20260809 12:57:41.982583 1469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:41.982846 1469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:41.983004 1469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:41.990703 1469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:41.990805 1469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:41.990984 1469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:57:42.000043 1469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:36345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:43115
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:42.001386 1469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:42.003213 1469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:42.016238 1475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:42.016275 1477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:42.016367 1474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:42.017086 1469 server_base.cc:1034] running on GCE node
I20260809 12:57:42.017678 1469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:42.018698 1469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:42.019888 1469 hybrid_clock.cc:648] HybridClock initialized: now 1786280262019854 us; error 34 us; skew 500 ppm
I20260809 12:57:42.020222 1469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:57:42.022500 1469 webserver.cc:460] Webserver started at http://127.21.235.67:42907/ using document root <none> and password file <none>
I20260809 12:57:42.023164 1469 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:42.023278 1469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:42.023607 1469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:42.026489 1469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "14a978447592443d95909f96c5ee64ef"
format_stamp: "Formatted at 2026-08-09 12:57:42 on dist-test-slave-gp6n"
server_key: "6269ae4a99901e214aa87728bf55e2d1"
server_key_iv: "09464231509e29942b7c1ae6e35c81b4"
server_key_version: "encryptionkey@0"
I20260809 12:57:42.027287 1469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "14a978447592443d95909f96c5ee64ef"
format_stamp: "Formatted at 2026-08-09 12:57:42 on dist-test-slave-gp6n"
server_key: "6269ae4a99901e214aa87728bf55e2d1"
server_key_iv: "09464231509e29942b7c1ae6e35c81b4"
server_key_version: "encryptionkey@0"
I20260809 12:57:42.032302 1469 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:57:42.035591 1483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:42.036835 1469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:57:42.037014 1469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "14a978447592443d95909f96c5ee64ef"
format_stamp: "Formatted at 2026-08-09 12:57:42 on dist-test-slave-gp6n"
server_key: "6269ae4a99901e214aa87728bf55e2d1"
server_key_iv: "09464231509e29942b7c1ae6e35c81b4"
server_key_version: "encryptionkey@0"
I20260809 12:57:42.037240 1469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:42.043570 1469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:42.044416 1469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:42.044714 1469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:42.045670 1469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:42.047300 1469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:42.047487 1469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:42.047627 1469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:42.047725 1469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:42.085085 1469 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:34977
I20260809 12:57:42.085143 1609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:34977 every 8 connection(s)
I20260809 12:57:42.086609 1469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:57:42.092487 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 1469
I20260809 12:57:42.092692 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:57:42.093140 22445 external_mini_cluster.cc:1428] Setting key 48438460b3ba340b60825d02957fc8fb
I20260809 12:57:42.095788 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:42.089451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:37565 (local address 127.21.235.126:43115)
0809 12:57:42.089649 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:42.089654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:42.091381 (+ 1727us) server_negotiation.cc:407] Connection header received
0809 12:57:42.093391 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:42.093397 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:42.093477 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:42.093589 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:42.094988 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:42.094993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:42.094998 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:57:42.095022 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:42.095060 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:57:42.095071 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:42.095194 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:42.095403 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 12:57:42.095472 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 12:57:42.096768 1613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:42.090045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:43115 (local address 127.21.235.67:37565)
0809 12:57:42.091213 (+ 1168us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:42.091270 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:42.093047 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:42.093819 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:42.093827 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:42.093925 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:42.094675 (+ 750us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:57:42.094677 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:42.094716 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:57:42.094805 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:42.095168 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:42.095179 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:42.095187 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 12:57:42.095482 (+ 295us) client_negotiation.cc:240] Negotiation successful
0809 12:57:42.095740 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":401,"threads_started":1}
I20260809 12:57:42.098073 1610 heartbeater.cc:347] Connected to a master server at 127.21.235.126:43115
I20260809 12:57:42.098461 1610 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:42.099087 1610 heartbeater.cc:511] Master 127.21.235.126:43115 requested a full tablet report, sending...
I20260809 12:57:42.100553 1106 ts_manager.cc:194] Registered new tserver with Master: 14a978447592443d95909f96c5ee64ef (127.21.235.67:34977)
I20260809 12:57:42.108743 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 12:57:42.114935 1318 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:57:42.113562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50972 (local address 127.21.235.126:43115)
0809 12:57:42.113743 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:42.113747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:42.114147 (+ 400us) server_negotiation.cc:407] Connection header received
0809 12:57:42.114281 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:42.114284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:42.114343 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:42.114482 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:42.114742 (+ 260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50972: BlockingRecv error: recv got EOF from 127.0.0.1:50972 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 12:57:42.115907 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 1174
I20260809 12:57:42.119640 1620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:42.116915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55518 (local address 127.21.235.65:34429)
0809 12:57:42.117545 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:42.117552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:42.117579 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:57:42.117636 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:42.117642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:42.117864 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:42.118057 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:42.119002 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:42.119008 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:42.119020 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:57:42.119039 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:42.119082 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:57:42.119097 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:42.119213 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:42.119265 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 12:57:42.119380 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":319,"threads_started":1}
I20260809 12:57:42.120879 1311 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:42.210959 1316 heartbeater.cc:503] Master 127.21.235.126:43115 was elected leader, sending a full tablet report...
I20260809 12:57:42.211849 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 1174
W20260809 12:57:42.232120 1091 connection.cc:535] server connection from 127.21.235.65:36213 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 12:57:42.232575 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 1320
I20260809 12:57:42.236789 1622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:42.233764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55642 (local address 127.21.235.66:34957)
0809 12:57:42.234367 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:42.234375 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:42.234457 (+ 82us) server_negotiation.cc:407] Connection header received
0809 12:57:42.234546 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:42.234552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:42.234815 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:42.234955 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:42.235989 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:42.235996 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:42.236009 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:57:42.236031 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:42.236082 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:57:42.236098 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:42.236209 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:42.236320 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 12:57:42.236481 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":277,"threads_started":1}
I20260809 12:57:42.238093 1459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:42.321048 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 1320
I20260809 12:57:42.339740 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 1469
I20260809 12:57:42.343992 1627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:42.340744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38236 (local address 127.21.235.67:34977)
0809 12:57:42.341471 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:42.341478 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:42.341509 (+ 31us) server_negotiation.cc:407] Connection header received
0809 12:57:42.341573 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:42.341579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:42.341802 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:42.341998 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:42.343234 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:42.343241 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:42.343252 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:57:42.343268 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:42.343307 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:57:42.343322 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:42.343525 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:42.343601 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 12:57:42.343701 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":385,"threads_started":1}
I20260809 12:57:42.345218 1605 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:42.429478 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 1469
I20260809 12:57:42.450250 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 1071
I20260809 12:57:42.451828 1141 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:42.561077 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 1071
2026-08-09T12:57:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2465 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:57:42 dist-test-slave-gp6n krb5kdc[1634](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:57:42 dist-test-slave-gp6n krb5kdc[1634](info): set up 2 sockets
Aug 09 12:57:42 dist-test-slave-gp6n krb5kdc[1634](info): commencing operation
krb5kdc: starting...
W20260809 12:57:43.752645 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.001s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:57:43 dist-test-slave-gp6n krb5kdc[1634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:57:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:57:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 12:57:44.031782 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:36565
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:36565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:57:44.374553 1648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:44.375020 1648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:44.375201 1648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:44.382381 1648 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:57:44.382550 1648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:44.382635 1648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:44.382722 1648 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:57:44.382789 1648 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:57:44.392659 1648 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:36565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:36565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:44.394061 1648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:44.395484 1648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:44.407668 1653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:44.407676 1654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:44.407848 1656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:44.408576 1648 server_base.cc:1034] running on GCE node
I20260809 12:57:44.409411 1648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:44.410938 1648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:44.412174 1648 hybrid_clock.cc:648] HybridClock initialized: now 1786280264412113 us; error 52 us; skew 500 ppm
I20260809 12:57:44.412588 1648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:44 dist-test-slave-gp6n krb5kdc[1634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280264, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:44.420104 1648 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:57:44.421801 1648 webserver.cc:460] Webserver started at http://127.21.235.126:45939/ using document root <none> and password file <none>
I20260809 12:57:44.422573 1648 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:44.422727 1648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:44.423043 1648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:44.426090 1648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "c027b2ba11ba451a9a8667a7d51c27dc"
format_stamp: "Formatted at 2026-08-09 12:57:44 on dist-test-slave-gp6n"
server_key: "8c30880851c427cedbb867f2c62346b8"
server_key_iv: "44b60ebfef30b8c67f632297ad0d0e2f"
server_key_version: "encryptionkey@0"
I20260809 12:57:44.426918 1648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "c027b2ba11ba451a9a8667a7d51c27dc"
format_stamp: "Formatted at 2026-08-09 12:57:44 on dist-test-slave-gp6n"
server_key: "8c30880851c427cedbb867f2c62346b8"
server_key_iv: "44b60ebfef30b8c67f632297ad0d0e2f"
server_key_version: "encryptionkey@0"
I20260809 12:57:44.432417 1648 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:57:44.436094 1663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:44.437470 1648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:57:44.437683 1648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "c027b2ba11ba451a9a8667a7d51c27dc"
format_stamp: "Formatted at 2026-08-09 12:57:44 on dist-test-slave-gp6n"
server_key: "8c30880851c427cedbb867f2c62346b8"
server_key_iv: "44b60ebfef30b8c67f632297ad0d0e2f"
server_key_version: "encryptionkey@0"
I20260809 12:57:44.437894 1648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:44.470319 1648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:44.474974 1648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:44.475306 1648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:44.495023 1648 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:36565
I20260809 12:57:44.495062 1721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:36565 every 8 connection(s)
I20260809 12:57:44.496438 1648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:57:44.499619 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 1648
I20260809 12:57:44.499876 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:57:44.500730 22445 external_mini_cluster.cc:1428] Setting key a61aa2227bee0de4f1924dd8ec096c92
I20260809 12:57:44.500811 1722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:44.514101 1722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc: Bootstrap starting.
Aug 09 12:57:44 dist-test-slave-gp6n krb5kdc[1634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:57:44.517573 1722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:44.519012 1722 log.cc:824] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc: Log is configured to *not* fsync() on all Append() calls
I20260809 12:57:44.522331 1722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc: No bootstrap required, opened a new log
I20260809 12:57:44.524974 1726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:44.502909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52290 (local address 127.21.235.126:36565)
0809 12:57:44.503752 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:44.503767 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:44.503807 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:57:44.505823 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:44.505845 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:44.506250 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:44.506711 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:44.508231 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:44.509512 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:44.511011 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:44.512123 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:44.512325 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:44.516362 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:44.516375 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:44.516392 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:44.516442 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:44.520412 (+ 3970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:44.521179 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:44.521187 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:44.521196 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:44.521411 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:44.521767 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:44.521772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:44.521775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:44.522492 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:44.522850 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:44.523150 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 12:57:44.523564 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":366,"threads_started":1}
I20260809 12:57:44.531454 1722 raft_consensus.cc:348] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c027b2ba11ba451a9a8667a7d51c27dc" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36565 } }
I20260809 12:57:44.531788 1722 raft_consensus.cc:374] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:44.531873 1722 raft_consensus.cc:729] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c027b2ba11ba451a9a8667a7d51c27dc, State: Initialized, Role: FOLLOWER
I20260809 12:57:44.532442 1722 consensus_queue.cc:260] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c027b2ba11ba451a9a8667a7d51c27dc" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36565 } }
I20260809 12:57:44.532644 1722 raft_consensus.cc:388] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:57:44.532740 1722 raft_consensus.cc:482] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:57:44.532866 1722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:44.535080 1722 raft_consensus.cc:504] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c027b2ba11ba451a9a8667a7d51c27dc" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36565 } }
I20260809 12:57:44.535522 1722 leader_election.cc:302] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c027b2ba11ba451a9a8667a7d51c27dc; no voters:
I20260809 12:57:44.536069 1722 leader_election.cc:288] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:57:44.536216 1730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:57:44.537010 1730 raft_consensus.cc:686] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc [term 1 LEADER]: Becoming Leader. State: Replica: c027b2ba11ba451a9a8667a7d51c27dc, State: Running, Role: LEADER
I20260809 12:57:44.537514 1730 consensus_queue.cc:237] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c027b2ba11ba451a9a8667a7d51c27dc" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36565 } }
I20260809 12:57:44.538710 1722 sys_catalog.cc:564] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:57:44.540870 1733 sys_catalog.cc:455] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c027b2ba11ba451a9a8667a7d51c27dc. Latest consensus state: current_term: 1 leader_uuid: "c027b2ba11ba451a9a8667a7d51c27dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c027b2ba11ba451a9a8667a7d51c27dc" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36565 } } }
I20260809 12:57:44.541169 1731 sys_catalog.cc:455] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c027b2ba11ba451a9a8667a7d51c27dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c027b2ba11ba451a9a8667a7d51c27dc" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36565 } } }
I20260809 12:57:44.541795 1731 sys_catalog.cc:458] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:44.542024 1733 sys_catalog.cc:458] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc [sys.catalog]: This master's current role is: LEADER
I20260809 12:57:44.547488 1741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:57:44.551684 1741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:57:44.560037 1741 catalog_manager.cc:1347] Generated new cluster ID: 15cf9416fa3a42aa9d26c3b83824ec54
I20260809 12:57:44.560160 1741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:57:44.572990 1741 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:57:44.574904 1741 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:57:44.581992 1741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc: Generated new TSK 0
I20260809 12:57:44.583050 1741 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
I20260809 12:57:44.655133 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:36565
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:57:44.997933 1755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:44.998208 1755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:44.998301 1755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:45.005527 1755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:45.005631 1755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:45.005744 1755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:57:45.015650 1755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:36565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:45.016924 1755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:45.018231 1755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:45.031597 1760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:45.031638 1761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:45.031836 1763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:45.032672 1755 server_base.cc:1034] running on GCE node
I20260809 12:57:45.033288 1755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:45.034256 1755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:45.035481 1755 hybrid_clock.cc:648] HybridClock initialized: now 1786280265035438 us; error 59 us; skew 500 ppm
I20260809 12:57:45.035804 1755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:45 dist-test-slave-gp6n krb5kdc[1634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280265, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:45.039145 1755 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 12:57:45.040562 1755 webserver.cc:460] Webserver started at http://127.21.235.65:44391/ using document root <none> and password file <none>
I20260809 12:57:45.041211 1755 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:45.041311 1755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:45.041647 1755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:45.044981 1755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "13e661b51f6f4dd19833101ac53efb54"
format_stamp: "Formatted at 2026-08-09 12:57:45 on dist-test-slave-gp6n"
server_key: "cb9844c156067a97e938e7ed84d78586"
server_key_iv: "2b669d2e303ec973be38367048b4592a"
server_key_version: "encryptionkey@0"
I20260809 12:57:45.045723 1755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "13e661b51f6f4dd19833101ac53efb54"
format_stamp: "Formatted at 2026-08-09 12:57:45 on dist-test-slave-gp6n"
server_key: "cb9844c156067a97e938e7ed84d78586"
server_key_iv: "2b669d2e303ec973be38367048b4592a"
server_key_version: "encryptionkey@0"
I20260809 12:57:45.050760 1755 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:57:45.054060 1771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:45.055526 1755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:57:45.055694 1755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "13e661b51f6f4dd19833101ac53efb54"
format_stamp: "Formatted at 2026-08-09 12:57:45 on dist-test-slave-gp6n"
server_key: "cb9844c156067a97e938e7ed84d78586"
server_key_iv: "2b669d2e303ec973be38367048b4592a"
server_key_version: "encryptionkey@0"
I20260809 12:57:45.055867 1755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:45.067734 1755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:45.071969 1755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:45.072278 1755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:45.073225 1755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:45.075106 1755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:45.075181 1755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:45.075321 1755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:45.075383 1755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:45.109030 1755 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:40755
I20260809 12:57:45.109067 1901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:40755 every 8 connection(s)
I20260809 12:57:45.110371 1755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:57:45.113126 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 1755
I20260809 12:57:45.113368 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:57:45.114174 22445 external_mini_cluster.cc:1428] Setting key e1b26eeb7c2c50bdc312cdc7aefdafac
Aug 09 12:57:45 dist-test-slave-gp6n krb5kdc[1634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280265, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:57:45.133090 1904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:45.113558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:57355 (local address 127.21.235.126:36565)
0809 12:57:45.114058 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:45.114063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:45.115263 (+ 1200us) server_negotiation.cc:407] Connection header received
0809 12:57:45.117645 (+ 2382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:45.117651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:45.117757 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:45.117882 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:45.119605 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:45.120411 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:45.122155 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:45.122674 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:45.122828 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:45.126860 (+ 4032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:45.126865 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:45.126869 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:45.126903 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:45.129566 (+ 2663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:45.130655 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:45.130659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:45.130661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:45.130782 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:45.131291 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:45.131293 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:45.131295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:45.131554 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:45.131665 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:45.132657 (+ 992us) server_negotiation.cc:299] Negotiation successful
0809 12:57:45.132836 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":261,"threads_started":1}
I20260809 12:57:45.134871 1905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:45.114164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:36565 (local address 127.21.235.65:57355)
0809 12:57:45.115067 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:45.115132 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:45.117364 (+ 2232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:45.118147 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:45.118156 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:45.118810 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:45.119420 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:45.119440 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:45.120606 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:45.120611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:45.121985 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:45.122003 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:45.122932 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:45.122942 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:45.123201 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:45.123945 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:45.123970 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:45.126587 (+ 2617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:45.129720 (+ 3133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:45.129733 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:45.129754 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:45.130528 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:45.130895 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:45.130903 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:45.130910 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:45.131176 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:45.131696 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:45.131708 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:45.132227 (+ 519us) client_negotiation.cc:769] Sending connection context
0809 12:57:45.132568 (+ 341us) client_negotiation.cc:240] Negotiation successful
0809 12:57:45.133188 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":348,"threads_started":1}
I20260809 12:57:45.136543 1902 heartbeater.cc:347] Connected to a master server at 127.21.235.126:36565
I20260809 12:57:45.136863 1902 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:45.137594 1902 heartbeater.cc:511] Master 127.21.235.126:36565 requested a full tablet report, sending...
I20260809 12:57:45.139933 1684 ts_manager.cc:194] Registered new tserver with Master: 13e661b51f6f4dd19833101ac53efb54 (127.21.235.65:40755)
I20260809 12:57:45.141657 1684 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:57355
WARNING: no policy specified for kudu/127.21.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
I20260809 12:57:45.181775 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:36565
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:57:45.525825 1908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:45.526153 1908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:45.526335 1908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:45.533671 1908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:45.533828 1908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:45.534000 1908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:57:45.544194 1908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:36565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:45.545570 1908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:45.547043 1908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:45.560643 1914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:45.560755 1913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:45.560672 1918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:45.561434 1908 server_base.cc:1034] running on GCE node
I20260809 12:57:45.561988 1908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:45.562971 1908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:45.564178 1908 hybrid_clock.cc:648] HybridClock initialized: now 1786280265564135 us; error 31 us; skew 500 ppm
I20260809 12:57:45.564507 1908 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:45 dist-test-slave-gp6n krb5kdc[1634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280265, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:45.568252 1908 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 12:57:45.570186 1908 webserver.cc:460] Webserver started at http://127.21.235.66:39615/ using document root <none> and password file <none>
I20260809 12:57:45.571180 1908 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:45.571323 1908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:45.571601 1908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:45.574496 1908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "0ebf990b1787416a8df1897cdea960ee"
format_stamp: "Formatted at 2026-08-09 12:57:45 on dist-test-slave-gp6n"
server_key: "fec2a9d026c7e19e0727649db557aba9"
server_key_iv: "435e21079c6728bfcdfe0a61d23fc189"
server_key_version: "encryptionkey@0"
I20260809 12:57:45.575254 1908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "0ebf990b1787416a8df1897cdea960ee"
format_stamp: "Formatted at 2026-08-09 12:57:45 on dist-test-slave-gp6n"
server_key: "fec2a9d026c7e19e0727649db557aba9"
server_key_iv: "435e21079c6728bfcdfe0a61d23fc189"
server_key_version: "encryptionkey@0"
I20260809 12:57:45.580673 1908 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 12:57:45.584481 1925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:45.585981 1908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:57:45.586169 1908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "0ebf990b1787416a8df1897cdea960ee"
format_stamp: "Formatted at 2026-08-09 12:57:45 on dist-test-slave-gp6n"
server_key: "fec2a9d026c7e19e0727649db557aba9"
server_key_iv: "435e21079c6728bfcdfe0a61d23fc189"
server_key_version: "encryptionkey@0"
I20260809 12:57:45.586382 1908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:45.606853 1908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:45.611318 1908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:45.611670 1908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:45.612812 1908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:45.614264 1908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:45.614354 1908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:45.614517 1908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:45.614576 1908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:45.651499 1908 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:39627
I20260809 12:57:45.651546 2057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:39627 every 8 connection(s)
I20260809 12:57:45.652871 1908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:57:45.659847 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 1908
I20260809 12:57:45.660061 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:57:45.660498 22445 external_mini_cluster.cc:1428] Setting key d4e883fa0cedcbb42d0d4eb79f7d8183
Aug 09 12:57:45 dist-test-slave-gp6n krb5kdc[1634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280265, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:57:45.673552 2060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:45.655685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:35071 (local address 127.21.235.126:36565)
0809 12:57:45.656145 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:45.656149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:45.657069 (+ 920us) server_negotiation.cc:407] Connection header received
0809 12:57:45.658944 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:45.658949 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:45.659018 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:45.659118 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:45.660938 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:45.661866 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:45.663655 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:45.664095 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:45.664229 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:45.667817 (+ 3588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:45.667822 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:45.667826 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:45.667865 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:45.670496 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:45.671197 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:45.671203 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:45.671206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:45.671339 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:45.671772 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:45.671776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:45.671778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:45.672053 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:45.672209 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:45.672922 (+ 713us) server_negotiation.cc:299] Negotiation successful
0809 12:57:45.673168 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":231,"threads_started":1}
I20260809 12:57:45.674573 2061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:45.656179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:36565 (local address 127.21.235.66:35071)
0809 12:57:45.656903 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:45.656948 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:45.658691 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:45.659314 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:45.659322 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:45.660060 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:45.660758 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:45.660779 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:45.662035 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:45.662039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:45.663439 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:45.663451 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:45.664339 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:45.664346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:45.664589 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:45.665275 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:45.665301 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:45.667587 (+ 2286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:45.670633 (+ 3046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:45.670641 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:45.670657 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:45.671068 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:45.671451 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:45.671456 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:45.671460 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:45.671647 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:45.672181 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:45.672188 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:45.672556 (+ 368us) client_negotiation.cc:769] Sending connection context
0809 12:57:45.672821 (+ 265us) client_negotiation.cc:240] Negotiation successful
0809 12:57:45.673361 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":266,"threads_started":1}
I20260809 12:57:45.676141 2058 heartbeater.cc:347] Connected to a master server at 127.21.235.126:36565
I20260809 12:57:45.676489 2058 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:45.677452 2058 heartbeater.cc:511] Master 127.21.235.126:36565 requested a full tablet report, sending...
I20260809 12:57:45.679380 1684 ts_manager.cc:194] Registered new tserver with Master: 0ebf990b1787416a8df1897cdea960ee (127.21.235.66:39627)
I20260809 12:57:45.680744 1684 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:35071
WARNING: no policy specified for kudu/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
I20260809 12:57:45.726480 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:36565
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:57:46.065562 2066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:57:46.065835 2066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:57:46.065930 2066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:57:46.073154 2066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:57:46.073261 2066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:57:46.073378 2066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:57:46.082888 2066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:36565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:57:46.084263 2066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:57:46.085538 2066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:57:46.099025 2072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:46.098940 2075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:57:46.098954 2073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:57:46.099682 2066 server_base.cc:1034] running on GCE node
I20260809 12:57:46.100238 2066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:57:46.101104 2066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:57:46.102294 2066 hybrid_clock.cc:648] HybridClock initialized: now 1786280266102219 us; error 78 us; skew 500 ppm
I20260809 12:57:46.102680 2066 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:57:46 dist-test-slave-gp6n krb5kdc[1634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280266, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:57:46.105857 2066 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 12:57:46.107513 2066 webserver.cc:460] Webserver started at http://127.21.235.67:34105/ using document root <none> and password file <none>
I20260809 12:57:46.108228 2066 fs_manager.cc:359] Metadata directory not provided
I20260809 12:57:46.108413 2066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:57:46.108773 2066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:57:46.112854 2066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "5f6c0a44629643fd86e7f6257940dc90"
format_stamp: "Formatted at 2026-08-09 12:57:46 on dist-test-slave-gp6n"
server_key: "730e375ed7a6d4bb01d7f71691b74400"
server_key_iv: "a63e38196b9bee2e859062849a26331c"
server_key_version: "encryptionkey@0"
I20260809 12:57:46.113777 2066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "5f6c0a44629643fd86e7f6257940dc90"
format_stamp: "Formatted at 2026-08-09 12:57:46 on dist-test-slave-gp6n"
server_key: "730e375ed7a6d4bb01d7f71691b74400"
server_key_iv: "a63e38196b9bee2e859062849a26331c"
server_key_version: "encryptionkey@0"
I20260809 12:57:46.119221 2066 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 12:57:46.122596 2082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:46.123860 2066 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:57:46.124044 2066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "5f6c0a44629643fd86e7f6257940dc90"
format_stamp: "Formatted at 2026-08-09 12:57:46 on dist-test-slave-gp6n"
server_key: "730e375ed7a6d4bb01d7f71691b74400"
server_key_iv: "a63e38196b9bee2e859062849a26331c"
server_key_version: "encryptionkey@0"
I20260809 12:57:46.124213 2066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:57:46.144989 1902 heartbeater.cc:503] Master 127.21.235.126:36565 was elected leader, sending a full tablet report...
I20260809 12:57:46.157024 2066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:57:46.161528 2066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:57:46.161831 2066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:57:46.163069 2066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:57:46.164552 2066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:57:46.164657 2066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:46.164821 2066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:57:46.164896 2066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:57:46.199522 2066 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:34683
I20260809 12:57:46.199572 2206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:34683 every 8 connection(s)
I20260809 12:57:46.200955 2066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:57:46.204190 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 2066
I20260809 12:57:46.204409 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:57:46.205032 22445 external_mini_cluster.cc:1428] Setting key 59241d74fd8cfe912bfddd3cbb9d6e2a
Aug 09 12:57:46 dist-test-slave-gp6n krb5kdc[1634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280266, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:57:46.224888 2212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:46.203714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:45207 (local address 127.21.235.126:36565)
0809 12:57:46.204175 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:46.204201 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:46.205956 (+ 1755us) server_negotiation.cc:407] Connection header received
0809 12:57:46.207476 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:46.207482 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:46.207597 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:46.207713 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:46.209341 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.210197 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.211807 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.212220 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.212355 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:46.215734 (+ 3379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:46.215740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:46.215745 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:46.215796 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:46.220275 (+ 4479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:46.221492 (+ 1217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:46.221498 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:46.221502 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:46.221648 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:46.222173 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:46.222178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:46.222182 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:46.222572 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:46.222722 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:46.224252 (+ 1530us) server_negotiation.cc:299] Negotiation successful
0809 12:57:46.224496 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":244,"threads_started":1}
I20260809 12:57:46.226748 2213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:46.204351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:36565 (local address 127.21.235.67:45207)
0809 12:57:46.205155 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:46.205211 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:46.207203 (+ 1992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:46.207995 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:46.208007 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:46.208566 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:57:46.209141 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:46.209159 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.210362 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.210367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:46.211656 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:46.211668 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.212450 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.212458 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:46.212699 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:46.213325 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:57:46.213351 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:57:46.215536 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:57:46.220528 (+ 4992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:46.220538 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:46.220613 (+ 75us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:46.221301 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:46.221774 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:57:46.221779 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:57:46.221782 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:57:46.222014 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:57:46.222794 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:57:46.222802 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:57:46.223244 (+ 442us) client_negotiation.cc:769] Sending connection context
0809 12:57:46.223587 (+ 343us) client_negotiation.cc:240] Negotiation successful
0809 12:57:46.225258 (+ 1671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":317,"threads_started":1}
I20260809 12:57:46.228384 2207 heartbeater.cc:347] Connected to a master server at 127.21.235.126:36565
I20260809 12:57:46.228734 2207 heartbeater.cc:464] Registering TS with master...
I20260809 12:57:46.229588 2207 heartbeater.cc:511] Master 127.21.235.126:36565 requested a full tablet report, sending...
I20260809 12:57:46.231460 1684 ts_manager.cc:194] Registered new tserver with Master: 5f6c0a44629643fd86e7f6257940dc90 (127.21.235.67:34683)
I20260809 12:57:46.232277 1684 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:45207
I20260809 12:57:46.243578 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:57:46.260977 2212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:46.248212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52296 (local address 127.21.235.126:36565)
0809 12:57:46.248383 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:46.248387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:46.248895 (+ 508us) server_negotiation.cc:407] Connection header received
0809 12:57:46.249109 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:46.249113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:46.249175 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:46.249288 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:46.250797 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.251523 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.252803 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.253219 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.253353 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:46.255752 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:46.255757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:46.255761 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:46.255791 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:46.258279 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:46.258997 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:46.259001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:46.259004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:46.259119 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:46.259521 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:46.259525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:46.259527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:46.259830 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:46.260107 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:46.260438 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 12:57:46.260666 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:57:46.265383 1684 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:57:46.272264 1684 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:57:46.296571 2225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:46.284551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:39627 (local address 127.0.0.1:60498)
0809 12:57:46.286028 (+ 1477us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:46.286047 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:46.286168 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:46.287932 (+ 1764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:46.287944 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:46.287986 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:46.288557 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:46.288569 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.290797 (+ 2228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.290805 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:46.293952 (+ 3147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:46.293969 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.295470 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.295477 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:46.295660 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:46.295714 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 12:57:46.295861 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 12:57:46.296054 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1340,"mutex_wait_us":25,"thread_start_us":310,"threads_started":1}
I20260809 12:57:46.296669 2230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:46.286319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:40755 (local address 127.0.0.1:35476)
0809 12:57:46.288568 (+ 2249us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:46.288587 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:46.288724 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:46.289615 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:46.289619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:46.289643 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:46.290005 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:46.290016 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.291964 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.291970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:46.294008 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:46.294021 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.295847 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.295855 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:46.296007 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:46.296038 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 12:57:46.296127 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 12:57:46.296256 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2126,"mutex_wait_us":43,"thread_start_us":1920,"threads_started":1}
I20260809 12:57:46.296991 2231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:46.286772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35476 (local address 127.21.235.65:40755)
0809 12:57:46.287652 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:46.287664 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:46.288709 (+ 1045us) server_negotiation.cc:407] Connection header received
0809 12:57:46.288860 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:46.288870 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:46.289087 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:46.289355 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:46.290196 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.291816 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.294165 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.295638 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.295781 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:46.296023 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:46.296460 (+ 437us) server_negotiation.cc:299] Negotiation successful
0809 12:57:46.296633 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":503,"threads_started":1}
I20260809 12:57:46.297729 2227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:46.284430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:34683 (local address 127.0.0.1:48684)
0809 12:57:46.285234 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:46.285271 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:46.285435 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:46.288899 (+ 3464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:46.288903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:46.288930 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:46.289396 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:46.289409 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.291756 (+ 2347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.291763 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:46.294366 (+ 2603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:46.294380 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.296183 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.296190 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:46.296400 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:46.296437 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 12:57:46.296539 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 12:57:46.297356 (+ 817us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":263,"thread_start_us":1306,"threads_started":1}
I20260809 12:57:46.297900 2229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:46.285586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60498 (local address 127.21.235.66:39627)
0809 12:57:46.287092 (+ 1506us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:46.287107 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:46.287144 (+ 37us) server_negotiation.cc:407] Connection header received
0809 12:57:46.287269 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:46.287282 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:46.287611 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:46.288796 (+ 1185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:46.289034 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.290673 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.294143 (+ 3470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.295255 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.295420 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:46.295643 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:46.297365 (+ 1722us) server_negotiation.cc:299] Negotiation successful
0809 12:57:46.297533 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1271,"thread_start_us":1313,"threads_started":1}
I20260809 12:57:46.298498 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:46.284793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48684 (local address 127.21.235.67:34683)
0809 12:57:46.288241 (+ 3448us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:46.288250 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:46.288288 (+ 38us) server_negotiation.cc:407] Connection header received
0809 12:57:46.288403 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:46.288409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:46.288663 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:46.289596 (+ 933us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:46.289878 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.291611 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.294527 (+ 2916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.295930 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.296114 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:46.296331 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:46.297972 (+ 1641us) server_negotiation.cc:299] Negotiation successful
0809 12:57:46.298140 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3257,"thread_start_us":3088,"threads_started":1}
I20260809 12:57:46.300782 1830 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4f711bb7a74e93898de0066d7c44be (DEFAULT_TABLE table=test-table [id=0a6de0ec41a24885bae18038bef44681]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:46.301558 1984 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4f711bb7a74e93898de0066d7c44be (DEFAULT_TABLE table=test-table [id=0a6de0ec41a24885bae18038bef44681]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:46.302192 1830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4f711bb7a74e93898de0066d7c44be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:46.301765 2136 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4f711bb7a74e93898de0066d7c44be (DEFAULT_TABLE table=test-table [id=0a6de0ec41a24885bae18038bef44681]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:57:46.303113 2136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4f711bb7a74e93898de0066d7c44be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:46.303248 1984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4f711bb7a74e93898de0066d7c44be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:57:46.313561 2232 tablet_bootstrap.cc:492] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54: Bootstrap starting.
I20260809 12:57:46.317749 2233 tablet_bootstrap.cc:492] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee: Bootstrap starting.
I20260809 12:57:46.318014 2232 tablet_bootstrap.cc:654] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:46.319110 2234 tablet_bootstrap.cc:492] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90: Bootstrap starting.
I20260809 12:57:46.319567 2232 log.cc:824] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54: Log is configured to *not* fsync() on all Append() calls
I20260809 12:57:46.321316 2233 tablet_bootstrap.cc:654] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:46.322239 2234 tablet_bootstrap.cc:654] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90: Neither blocks nor log segments found. Creating new log.
I20260809 12:57:46.322599 2233 log.cc:824] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee: Log is configured to *not* fsync() on all Append() calls
I20260809 12:57:46.323063 2232 tablet_bootstrap.cc:492] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54: No bootstrap required, opened a new log
I20260809 12:57:46.323390 2232 ts_tablet_manager.cc:1397] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260809 12:57:46.323385 2234 log.cc:824] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90: Log is configured to *not* fsync() on all Append() calls
I20260809 12:57:46.326210 2234 tablet_bootstrap.cc:492] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90: No bootstrap required, opened a new log
I20260809 12:57:46.326463 2233 tablet_bootstrap.cc:492] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee: No bootstrap required, opened a new log
I20260809 12:57:46.326560 2234 ts_tablet_manager.cc:1397] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260809 12:57:46.326725 2233 ts_tablet_manager.cc:1397] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260809 12:57:46.333952 2234 raft_consensus.cc:348] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6c0a44629643fd86e7f6257940dc90" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34683 } } peers { permanent_uuid: "0ebf990b1787416a8df1897cdea960ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39627 } } peers { permanent_uuid: "13e661b51f6f4dd19833101ac53efb54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40755 } }
I20260809 12:57:46.334338 2234 raft_consensus.cc:374] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:46.334445 2234 raft_consensus.cc:729] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f6c0a44629643fd86e7f6257940dc90, State: Initialized, Role: FOLLOWER
I20260809 12:57:46.334227 2232 raft_consensus.cc:348] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6c0a44629643fd86e7f6257940dc90" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34683 } } peers { permanent_uuid: "0ebf990b1787416a8df1897cdea960ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39627 } } peers { permanent_uuid: "13e661b51f6f4dd19833101ac53efb54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40755 } }
I20260809 12:57:46.334663 2232 raft_consensus.cc:374] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:46.334755 2232 raft_consensus.cc:729] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13e661b51f6f4dd19833101ac53efb54, State: Initialized, Role: FOLLOWER
I20260809 12:57:46.335103 2234 consensus_queue.cc:260] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6c0a44629643fd86e7f6257940dc90" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34683 } } peers { permanent_uuid: "0ebf990b1787416a8df1897cdea960ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39627 } } peers { permanent_uuid: "13e661b51f6f4dd19833101ac53efb54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40755 } }
I20260809 12:57:46.335372 2232 consensus_queue.cc:260] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6c0a44629643fd86e7f6257940dc90" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34683 } } peers { permanent_uuid: "0ebf990b1787416a8df1897cdea960ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39627 } } peers { permanent_uuid: "13e661b51f6f4dd19833101ac53efb54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40755 } }
I20260809 12:57:46.337270 2207 heartbeater.cc:503] Master 127.21.235.126:36565 was elected leader, sending a full tablet report...
I20260809 12:57:46.338008 2234 ts_tablet_manager.cc:1428] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20260809 12:57:46.338351 2232 ts_tablet_manager.cc:1428] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260809 12:57:46.339286 2233 raft_consensus.cc:348] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6c0a44629643fd86e7f6257940dc90" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34683 } } peers { permanent_uuid: "0ebf990b1787416a8df1897cdea960ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39627 } } peers { permanent_uuid: "13e661b51f6f4dd19833101ac53efb54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40755 } }
I20260809 12:57:46.339740 2233 raft_consensus.cc:374] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:57:46.339854 2233 raft_consensus.cc:729] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ebf990b1787416a8df1897cdea960ee, State: Initialized, Role: FOLLOWER
I20260809 12:57:46.340425 2233 consensus_queue.cc:260] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6c0a44629643fd86e7f6257940dc90" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34683 } } peers { permanent_uuid: "0ebf990b1787416a8df1897cdea960ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39627 } } peers { permanent_uuid: "13e661b51f6f4dd19833101ac53efb54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40755 } }
I20260809 12:57:46.341630 2058 heartbeater.cc:503] Master 127.21.235.126:36565 was elected leader, sending a full tablet report...
I20260809 12:57:46.342269 2233 ts_tablet_manager.cc:1428] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee: Time spent starting tablet: real 0.015s user 0.011s sys 0.002s
W20260809 12:57:46.362865 1903 tablet.cc:2369] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:57:46.405093 2059 tablet.cc:2369] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:57:46.452780 2210 tablet.cc:2369] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:57:46.457005 2239 raft_consensus.cc:482] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:57:46.457266 2239 raft_consensus.cc:504] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6c0a44629643fd86e7f6257940dc90" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34683 } } peers { permanent_uuid: "0ebf990b1787416a8df1897cdea960ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39627 } } peers { permanent_uuid: "13e661b51f6f4dd19833101ac53efb54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40755 } }
I20260809 12:57:46.458998 2239 leader_election.cc:288] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f6c0a44629643fd86e7f6257940dc90 (127.21.235.67:34683), 0ebf990b1787416a8df1897cdea960ee (127.21.235.66:39627)
I20260809 12:57:46.467991 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:46.459527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:35315 (local address 127.21.235.67:34683)
0809 12:57:46.459779 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:46.459785 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:46.460938 (+ 1153us) server_negotiation.cc:407] Connection header received
0809 12:57:46.461164 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:46.461170 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:46.461241 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:46.461368 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:46.461836 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.463011 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.465350 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.466803 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.466983 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:46.467069 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:46.467551 (+ 482us) server_negotiation.cc:299] Negotiation successful
0809 12:57:46.467710 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260809 12:57:46.468146 2229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:46.459598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:35083 (local address 127.21.235.66:39627)
0809 12:57:46.459793 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:46.459801 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:46.460356 (+ 555us) server_negotiation.cc:407] Connection header received
0809 12:57:46.460672 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:46.460696 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:46.460783 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:46.460882 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:57:46.461452 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.462708 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.465326 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.466794 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.466983 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:46.467066 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:46.467551 (+ 485us) server_negotiation.cc:299] Negotiation successful
0809 12:57:46.467873 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:57:46.468173 2242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:46.459582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:34683 (local address 127.21.235.65:35315)
0809 12:57:46.460805 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:46.460828 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:46.461008 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:46.461386 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:46.461393 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:46.461419 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:46.461691 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:46.461698 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.463141 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.463147 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:46.465173 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:46.465188 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.467106 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.467111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:46.467328 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:46.467393 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 12:57:46.467494 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 12:57:46.467785 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1120,"mutex_wait_us":156,"thread_start_us":872,"threads_started":1}
I20260809 12:57:46.468551 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:46.459428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:39627 (local address 127.21.235.65:35083)
0809 12:57:46.460229 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0809 12:57:46.460246 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:57:46.460492 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:57:46.460898 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:57:46.460902 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:57:46.460930 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:57:46.461285 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:46.461294 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.462831 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.462835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:46.465087 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:57:46.465105 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.467093 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.467100 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:57:46.467341 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:46.467400 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 12:57:46.467498 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 12:57:46.467717 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":97,"thread_start_us":502,"threads_started":1}
I20260809 12:57:46.469501 2157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4f711bb7a74e93898de0066d7c44be" candidate_uuid: "13e661b51f6f4dd19833101ac53efb54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f6c0a44629643fd86e7f6257940dc90" is_pre_election: true
I20260809 12:57:46.469600 2006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4f711bb7a74e93898de0066d7c44be" candidate_uuid: "13e661b51f6f4dd19833101ac53efb54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ebf990b1787416a8df1897cdea960ee" is_pre_election: true
I20260809 12:57:46.470046 2157 raft_consensus.cc:2454] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13e661b51f6f4dd19833101ac53efb54 in term 0.
I20260809 12:57:46.470196 2006 raft_consensus.cc:2454] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13e661b51f6f4dd19833101ac53efb54 in term 0.
I20260809 12:57:46.470779 1774 leader_election.cc:302] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13e661b51f6f4dd19833101ac53efb54, 5f6c0a44629643fd86e7f6257940dc90; no voters:
I20260809 12:57:46.471239 2239 raft_consensus.cc:2781] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:57:46.471395 2239 raft_consensus.cc:482] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:57:46.471524 2239 raft_consensus.cc:3037] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:46.473677 2239 raft_consensus.cc:504] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6c0a44629643fd86e7f6257940dc90" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34683 } } peers { permanent_uuid: "0ebf990b1787416a8df1897cdea960ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39627 } } peers { permanent_uuid: "13e661b51f6f4dd19833101ac53efb54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40755 } }
I20260809 12:57:46.474298 2239 leader_election.cc:288] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [CANDIDATE]: Term 1 election: Requested vote from peers 5f6c0a44629643fd86e7f6257940dc90 (127.21.235.67:34683), 0ebf990b1787416a8df1897cdea960ee (127.21.235.66:39627)
I20260809 12:57:46.474853 2157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4f711bb7a74e93898de0066d7c44be" candidate_uuid: "13e661b51f6f4dd19833101ac53efb54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f6c0a44629643fd86e7f6257940dc90"
I20260809 12:57:46.474943 2006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4f711bb7a74e93898de0066d7c44be" candidate_uuid: "13e661b51f6f4dd19833101ac53efb54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ebf990b1787416a8df1897cdea960ee"
I20260809 12:57:46.475031 2157 raft_consensus.cc:3037] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:46.475260 2006 raft_consensus.cc:3037] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:57:46.477702 2157 raft_consensus.cc:2454] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13e661b51f6f4dd19833101ac53efb54 in term 1.
I20260809 12:57:46.478124 1774 leader_election.cc:302] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13e661b51f6f4dd19833101ac53efb54, 5f6c0a44629643fd86e7f6257940dc90; no voters:
I20260809 12:57:46.478257 2006 raft_consensus.cc:2454] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13e661b51f6f4dd19833101ac53efb54 in term 1.
I20260809 12:57:46.478469 2239 raft_consensus.cc:2781] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:57:46.479048 2239 raft_consensus.cc:686] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [term 1 LEADER]: Becoming Leader. State: Replica: 13e661b51f6f4dd19833101ac53efb54, State: Running, Role: LEADER
I20260809 12:57:46.479665 2239 consensus_queue.cc:237] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6c0a44629643fd86e7f6257940dc90" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34683 } } peers { permanent_uuid: "0ebf990b1787416a8df1897cdea960ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39627 } } peers { permanent_uuid: "13e661b51f6f4dd19833101ac53efb54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40755 } }
I20260809 12:57:46.484598 1682 catalog_manager.cc:5495] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13e661b51f6f4dd19833101ac53efb54 (127.21.235.65). New cstate: current_term: 1 leader_uuid: "13e661b51f6f4dd19833101ac53efb54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6c0a44629643fd86e7f6257940dc90" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ebf990b1787416a8df1897cdea960ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13e661b51f6f4dd19833101ac53efb54" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40755 } health_report { overall_health: HEALTHY } } }
I20260809 12:57:46.535260 2231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:46.529871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35484 (local address 127.21.235.65:40755)
0809 12:57:46.530165 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:46.530170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:46.530195 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:57:46.530378 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:46.530382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:46.530530 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:46.530673 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:57:46.531015 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.531788 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.533057 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.533514 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.533614 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:46.534046 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:57:46.534647 (+ 601us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:57:46.534772 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:46.534872 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 12:57:46.534938 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260809 12:57:46.543817 2157 raft_consensus.cc:1261] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90 [term 1 FOLLOWER]: Refusing update from remote peer 13e661b51f6f4dd19833101ac53efb54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:57:46.544131 2006 raft_consensus.cc:1261] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee [term 1 FOLLOWER]: Refusing update from remote peer 13e661b51f6f4dd19833101ac53efb54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:57:46.544881 2243 consensus_queue.cc:1035] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f6c0a44629643fd86e7f6257940dc90" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 34683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:57:46.545310 2239 consensus_queue.cc:1035] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ebf990b1787416a8df1897cdea960ee" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:57:46.571905 1683 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:57:46.572193 1683 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:57:46.575862 1683 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c027b2ba11ba451a9a8667a7d51c27dc: Sending DeleteTablet for 3 replicas of tablet 8a4f711bb7a74e93898de0066d7c44be
I20260809 12:57:46.577001 2136 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4f711bb7a74e93898de0066d7c44be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:46 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:48684
I20260809 12:57:46.577226 1984 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4f711bb7a74e93898de0066d7c44be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:46 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:60498
I20260809 12:57:46.577227 1830 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4f711bb7a74e93898de0066d7c44be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:57:46 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:35476
I20260809 12:57:46.580034 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 1755
I20260809 12:57:46.589723 2258 tablet_replica.cc:330] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee: stopping tablet replica
I20260809 12:57:46.590327 2258 raft_consensus.cc:2229] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:57:46.591001 2258 raft_consensus.cc:2258] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:46.591948 2259 tablet_replica.cc:330] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54: stopping tablet replica
I20260809 12:57:46.592315 2259 raft_consensus.cc:2229] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:57:46.592909 2259 raft_consensus.cc:2258] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:46.593868 2257 tablet_replica.cc:330] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90: stopping tablet replica
I20260809 12:57:46.594198 2257 raft_consensus.cc:2229] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:57:46.594671 2257 raft_consensus.cc:2258] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:57:46.596065 2258 ts_tablet_manager.cc:1905] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:46.598455 2259 ts_tablet_manager.cc:1905] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:46.603123 2258 ts_tablet_manager.cc:1918] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:57:46.603300 2258 log.cc:1196] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/8a4f711bb7a74e93898de0066d7c44be
I20260809 12:57:46.603859 2258 ts_tablet_manager.cc:1939] T 8a4f711bb7a74e93898de0066d7c44be P 0ebf990b1787416a8df1897cdea960ee: Deleting consensus metadata
I20260809 12:57:46.604628 2257 ts_tablet_manager.cc:1905] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:57:46.605716 1669 catalog_manager.cc:4841] TS 0ebf990b1787416a8df1897cdea960ee (127.21.235.66:39627): tablet 8a4f711bb7a74e93898de0066d7c44be (table test-table [id=0a6de0ec41a24885bae18038bef44681]) successfully deleted
I20260809 12:57:46.606550 2259 ts_tablet_manager.cc:1918] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:57:46.606734 2259 log.cc:1196] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/8a4f711bb7a74e93898de0066d7c44be
I20260809 12:57:46.607261 2259 ts_tablet_manager.cc:1939] T 8a4f711bb7a74e93898de0066d7c44be P 13e661b51f6f4dd19833101ac53efb54: Deleting consensus metadata
Aug 09 12:57:46 dist-test-slave-gp6n krb5kdc[1634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 12:57:46.609064 1667 catalog_manager.cc:4841] TS 13e661b51f6f4dd19833101ac53efb54 (127.21.235.65:40755): tablet 8a4f711bb7a74e93898de0066d7c44be (table test-table [id=0a6de0ec41a24885bae18038bef44681]) successfully deleted
I20260809 12:57:46.611235 2257 ts_tablet_manager.cc:1918] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:57:46.611410 2257 log.cc:1196] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/8a4f711bb7a74e93898de0066d7c44be
I20260809 12:57:46.611960 2257 ts_tablet_manager.cc:1939] T 8a4f711bb7a74e93898de0066d7c44be P 5f6c0a44629643fd86e7f6257940dc90: Deleting consensus metadata
I20260809 12:57:46.613813 1665 catalog_manager.cc:4841] TS 5f6c0a44629643fd86e7f6257940dc90 (127.21.235.67:34683): tablet 8a4f711bb7a74e93898de0066d7c44be (table test-table [id=0a6de0ec41a24885bae18038bef44681]) successfully deleted
I20260809 12:57:46.616189 2231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:46.581514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35492 (local address 127.21.235.65:40755)
0809 12:57:46.581741 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:46.581750 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:46.596739 (+ 14989us) server_negotiation.cc:407] Connection header received
0809 12:57:46.596951 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:46.596958 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:46.597053 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:46.597165 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:46.599062 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.600256 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.602172 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.602856 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.602968 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:46.608823 (+ 5855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:46.608837 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:46.608855 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:46.608903 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:46.613035 (+ 4132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:46.613951 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:46.613961 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:46.613964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:46.614156 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:46.614635 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:46.614638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:46.614640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:46.614994 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:46.615206 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:46.615636 (+ 430us) server_negotiation.cc:299] Negotiation successful
0809 12:57:46.615838 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":68}
I20260809 12:57:46.616882 1897 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:46.726135 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 1755
I20260809 12:57:46.748270 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 1908
Aug 09 12:57:46 dist-test-slave-gp6n krb5kdc[1634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.66@KRBTEST.COM
I20260809 12:57:46.762949 2229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:46.748983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60506 (local address 127.21.235.66:39627)
0809 12:57:46.749152 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:46.749156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:46.749175 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:57:46.749344 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:46.749348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:46.749427 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:46.749512 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:46.750795 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.751534 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.752710 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.753083 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.753172 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:46.757292 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:46.757307 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:46.757323 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:46.757359 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:46.760327 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:46.760976 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:46.760982 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:46.760985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:46.761105 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:46.761518 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:46.761521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:46.761523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:46.761926 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:46.762150 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:46.762502 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 12:57:46.762669 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 12:57:46.763602 2053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:46.854151 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 1908
I20260809 12:57:46.872643 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 2066
Aug 09 12:57:46 dist-test-slave-gp6n krb5kdc[1634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.67@KRBTEST.COM
I20260809 12:57:46.888245 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:57:46.873490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48694 (local address 127.21.235.67:34683)
0809 12:57:46.873693 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 12:57:46.873699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:57:46.873721 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:57:46.873829 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:57:46.873833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:57:46.873897 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:57:46.874018 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:57:46.875391 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.876114 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.877345 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:57:46.877745 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:57:46.877884 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:57:46.882490 (+ 4606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:57:46.882505 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:57:46.882521 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:57:46.882557 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:57:46.885391 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:46.886080 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:46.886093 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:46.886096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:46.886246 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:57:46.886663 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:57:46.886668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:57:46.886671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:57:46.887037 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:57:46.887386 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0809 12:57:46.887691 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 12:57:46.887910 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:57:46.888976 2201 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:46.979503 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 2066
I20260809 12:57:46.998639 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 1648
I20260809 12:57:46.999873 1717 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:57:47.117592 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 1648
2026-08-09T12:57:47Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4559 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:57:47 dist-test-slave-gp6n krb5kdc[2267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:57:47 dist-test-slave-gp6n krb5kdc[2267](info): set up 2 sockets
Aug 09 12:57:47 dist-test-slave-gp6n krb5kdc[2267](info): commencing operation
krb5kdc: starting...
W20260809 12:57:48.418277 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:57:48 dist-test-slave-gp6n krb5kdc[2267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:57:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:57:48Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.235.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.235.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.235.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/rangeradmin_127.21.235.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.235.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/rangeradmin_127.21.235.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.235.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.235.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.235.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/rangerlookup_127.21.235.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.235.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/rangerlookup_127.21.235.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.235.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.235.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.235.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/HTTP_127.21.235.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.235.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/HTTP_127.21.235.84@KRBTEST.COM.keytab.
I20260809 12:57:48.809543 22445 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskjg7NUQ/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/postgres -l logfile start
2026-08-09 12:57:51.527 UTC [2300] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 12:57:51.527 UTC [2300] LOG: listening on IPv4 address "127.21.235.84", port 46057
2026-08-09 12:57:51.535 UTC [2300] LOG: listening on Unix socket "/tmp/.s.PGSQL.46057"
2026-08-09 12:57:51.563 UTC [2303] LOG: database system was shut down at 2026-08-09 12:57:49 UTC
2026-08-09 12:57:51.567 UTC [2300] LOG: database system is ready to accept connections
I20260809 12:57:52.651085 22445 mini_postgres.cc:96] Postgres bound to 46057
2026-08-09 12:57:52.676 UTC [2311] FATAL: database "slave" does not exist
127.21.235.84:46057 - accepting connections
I20260809 12:57:52.677585 22445 mini_ranger.cc:161] Starting Ranger...
I20260809 12:57:52.712168 22445 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 12:57:53.259797 22445 mini_ranger.cc:87] Created ranger Postgres database
I20260809 12:57:53.259943 22445 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ranger-admin
2026-08-09 12:57:53,597 [I] DB FLAVOR :POSTGRES
2026-08-09 12:57:53,597 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 12:57:53,597 [I] Checking connection..
2026-08-09 12:57:53,597 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 12:57:53,865 [I] Checking connection passed.
2026-08-09 12:57:53,866 [I] --------- Verifying version history table ---------
2026-08-09 12:57:53,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 12:57:54,132 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 12:57:54,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 12:57:54,381 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 12:57:54,381 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 12:57:54,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 12:57:54.635 UTC [2385] WARNING: there is no transaction in progress
2026-08-09 12:57:54,643 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 12:57:54,644 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 12:57:54,904 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 12:57:54,905 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 12:57:54,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 12:57:55,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 12:57:55,417 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gp6n.c.gcp-upstream.internal','N') ;"
2026-08-09 12:57:55,650 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 12:57:55,915 [I] Table x_portal_user does not exist in database ranger
2026-08-09 12:57:55,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 12:57:56,165 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 12:57:56,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 12:57:56,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 12:57:56,663 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 12:57:56,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 12:57:57.659 UTC [2512] WARNING: there is no transaction in progress
2026-08-09 12:57:57.675 UTC [2512] WARNING: there is no transaction in progress
2026-08-09 12:57:57.691 UTC [2512] WARNING: there is no transaction in progress
2026-08-09 12:57:57.706 UTC [2512] WARNING: there is no transaction in progress
2026-08-09 12:57:57.724 UTC [2512] WARNING: there is no transaction in progress
2026-08-09 12:57:57.746 UTC [2512] WARNING: there is no transaction in progress
2026-08-09 12:57:57.811 UTC [2512] WARNING: there is no transaction in progress
2026-08-09 12:57:57.818 UTC [2512] WARNING: there is no transaction in progress
2026-08-09 12:57:57.833 UTC [2512] WARNING: there is no transaction in progress
2026-08-09 12:57:57.843 UTC [2512] WARNING: there is no transaction in progress
2026-08-09 12:57:57.853 UTC [2512] WARNING: there is no transaction in progress
2026-08-09 12:57:57.867 UTC [2512] WARNING: there is no transaction in progress
2026-08-09 12:57:57.877 UTC [2512] WARNING: there is no transaction in progress
2026-08-09 12:57:57.883 UTC [2512] WARNING: there is no transaction in progress
2026-08-09 12:57:57.890 UTC [2512] WARNING: there is no transaction in progress
2026-08-09 12:57:57.898 UTC [2512] WARNING: there is no transaction in progress
2026-08-09 12:57:58,584 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 12:57:58,584 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 12:57:58,839 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 12:57:58,840 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 12:57:59,084 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 12:57:59,084 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 12:57:59,350 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 12:57:59,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 12:57:59,606 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 12:57:59,848 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gp6n.c.gcp-upstream.internal';"
2026-08-09 12:58:00,095 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 12:58:00,096 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 12:58:00,357 [I] DB_PATCHES have already been applied
I20260809 12:58:00.363884 22445 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ranger-admin:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskjg7NUQ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 12:58:00.364022 22445 mini_ranger.cc:193] Using host: 127.21.235.84
I20260809 12:58:00.385126 22445 mini_ranger.cc:227] Ranger admin URL: http://127.21.235.84:46999
Aug 09, 2026 12:58:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:58:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:58:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 12:58:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 12:58:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 12:58:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 12:58:01 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 12:58:01.621707 22445 mini_ranger.cc:160] Time spent starting Ranger: real 8.944s user 0.003s sys 0.061s
Aug 09, 2026 12:58:22 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 12:58:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 09, 2026 12:58:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 12:58:22 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 12:58:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 12:58:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 12:58:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 12:58:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 12:58:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 12:58:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 12:58:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 12:58:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 12:58:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 12:58:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 12:58:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 12:58:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 12:58:22 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 12:58:23 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 12:58:24.240604 22445 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.235.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.235.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.235.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/rangerkms_127.21.235.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.235.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/rangerkms_127.21.235.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.235.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.235.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.235.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/HTTP_127.21.235.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.235.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/HTTP_127.21.235.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 12:58:24 dist-test-slave-gp6n krb5kdc[2267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280304, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 12:58:24.453709 22445 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 12:58:24.483398 22445 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 12:58:25.410560 22445 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 12:58:25.410806 22445 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ranger-kms
I20260809 12:58:25.410913 22445 mini_ranger_kms.cc:227] Using postgres at 127.21.235.84:46057
2026-08-09 12:58:25,923 [I] DB FLAVOR :POSTGRES
2026-08-09 12:58:25,924 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 12:58:25,924 [I] Checking connection
2026-08-09 12:58:25,924 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 12:58:26,173 [I] connection success
2026-08-09 12:58:26,174 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 12:58:26,174 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 12:58:26,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 12:58:26,427 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 12:58:26,427 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 12:58:26,427 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 12:58:26,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:46057/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 12:58:26,691 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 12:58:27.208827 22445 mini_ranger_kms.cc:310] Created kms service
I20260809 12:58:27.454033 22445 mini_ranger_kms.cc:326] Created kudu user
I20260809 12:58:27.504701 22445 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 12:58:27 dist-test-slave-gp6n krb5kdc[2267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280304, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
I20260809 12:58:27.964401 22445 mini_ranger_kms.cc:384] Added ranger policy
I20260809 12:58:27.964826 22445 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ranger-kms:/tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 12:58:27.964897 22445 mini_ranger_kms.cc:240] Using host: 127.21.235.84
I20260809 12:58:27.996744 22445 mini_ranger_kms.cc:276] Ranger KMS PID: 2719
I20260809 12:58:27.996951 22445 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.235.84:54527
Aug 09, 2026 12:58:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:58:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:58:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 12:58:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 12:58:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 12:58:29 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 12:58:30.555625 22445 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.102s user 0.128s sys 0.430s
I20260809 12:58:30.555954 22445 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 12:58:30.556181 22445 mini_ranger_kms.cc:401] 127.21.235.84:54527/kms/v1/keys
Aug 09 12:58:30 dist-test-slave-gp6n krb5kdc[2267](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786280310, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.235.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 12:58:31 dist-test-slave-gp6n krb5kdc[2267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280310, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.235.84@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
Aug 09 12:58:31 dist-test-slave-gp6n krb5kdc[2267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280310, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.235.84@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
Aug 09 12:58:33 dist-test-slave-gp6n krb5kdc[2267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 12:58:33.855247 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:43295
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:43295
--ranger_config_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.235.84:54527/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:58:34.230813 2775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:58:34.231093 2775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:58:34.231329 2775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:58:34.238726 2775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:58:34.238837 2775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:58:34.238917 2775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:58:34.239022 2775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:58:34.239104 2775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:58:34.249541 2775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.235.84:54527/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:43295
--ranger_config_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:43295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:58:34.250872 2775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:58:34.252213 2775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:58:34.264945 2781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:58:34.264968 2783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:58:34.265743 2780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:58:34.266140 2775 server_base.cc:1034] running on GCE node
I20260809 12:58:34.267007 2775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:58:34.268414 2775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:58:34.269654 2775 hybrid_clock.cc:648] HybridClock initialized: now 1786280314269609 us; error 38 us; skew 500 ppm
I20260809 12:58:34.270093 2775 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:58:34 dist-test-slave-gp6n krb5kdc[2267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280314, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:58:34.273496 2775 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:58:34.275094 2775 webserver.cc:460] Webserver started at http://127.21.235.126:44527/ using document root <none> and password file <none>
I20260809 12:58:34.275759 2775 fs_manager.cc:359] Metadata directory not provided
I20260809 12:58:34.275910 2775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:58:34.276228 2775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:58:34 dist-test-slave-gp6n krb5kdc[2267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280314, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
I20260809 12:58:34.333751 2775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "421ddb0884a04dd8ab35b67c73c44e0a"
format_stamp: "Formatted at 2026-08-09 12:58:34 on dist-test-slave-gp6n"
server_key: "c6fd319b2a0e9122cff32ec5efe79c00"
server_key_iv: "5bedb2821c4e7394ba564f0c76ca669a"
server_key_version: "kuduclusterkey@0"
I20260809 12:58:34.334712 2775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "421ddb0884a04dd8ab35b67c73c44e0a"
format_stamp: "Formatted at 2026-08-09 12:58:34 on dist-test-slave-gp6n"
server_key: "c6fd319b2a0e9122cff32ec5efe79c00"
server_key_iv: "5bedb2821c4e7394ba564f0c76ca669a"
server_key_version: "kuduclusterkey@0"
I20260809 12:58:34.340683 2775 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260809 12:58:34.359025 2795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:58:34.360598 2775 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260809 12:58:34.360812 2775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "421ddb0884a04dd8ab35b67c73c44e0a"
format_stamp: "Formatted at 2026-08-09 12:58:34 on dist-test-slave-gp6n"
server_key: "c6fd319b2a0e9122cff32ec5efe79c00"
server_key_iv: "5bedb2821c4e7394ba564f0c76ca669a"
server_key_version: "kuduclusterkey@0"
I20260809 12:58:34.361057 2775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:58:34.407940 2775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:58:34.412221 2775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:58:34.412585 2775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:58:34.434576 2775 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:43295
I20260809 12:58:34.434666 2851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:43295 every 8 connection(s)
I20260809 12:58:34.436074 2775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:58:34.437760 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 2775
I20260809 12:58:34.438011 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:58:34.438946 22445 external_mini_cluster.cc:1428] Setting key ecd71bb10024bb08e5d904efc5cdb62a
I20260809 12:58:34.441758 2852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 12:58:34 dist-test-slave-gp6n krb5kdc[2267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:58:34.456157 2852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a: Bootstrap starting.
I20260809 12:58:34.460661 2852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a: Neither blocks nor log segments found. Creating new log.
I20260809 12:58:34.462015 2852 log.cc:824] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a: Log is configured to *not* fsync() on all Append() calls
I20260809 12:58:34.465570 2855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:34.441305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37316 (local address 127.21.235.126:43295)
0809 12:58:34.442278 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0809 12:58:34.442294 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 12:58:34.442348 (+ 54us) server_negotiation.cc:407] Connection header received
0809 12:58:34.444860 (+ 2512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:58:34.444888 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:58:34.445435 (+ 547us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:58:34.445906 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:58:34.447273 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:34.449767 (+ 2494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:34.451212 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:34.452325 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:34.452519 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:34.456627 (+ 4108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:58:34.456639 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:58:34.456658 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:58:34.456720 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:58:34.461012 (+ 4292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:58:34.461715 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:58:34.461724 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:58:34.461735 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:58:34.461970 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:58:34.462385 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:58:34.462390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:58:34.462393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:58:34.463274 (+ 881us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:58:34.463603 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 12:58:34.464105 (+ 502us) server_negotiation.cc:299] Negotiation successful
0809 12:58:34.464547 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":469,"threads_started":1}
I20260809 12:58:34.465827 2852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a: No bootstrap required, opened a new log
I20260809 12:58:34.473850 2852 raft_consensus.cc:348] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "421ddb0884a04dd8ab35b67c73c44e0a" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43295 } }
I20260809 12:58:34.474265 2852 raft_consensus.cc:374] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:58:34.474442 2852 raft_consensus.cc:729] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 421ddb0884a04dd8ab35b67c73c44e0a, State: Initialized, Role: FOLLOWER
I20260809 12:58:34.475077 2852 consensus_queue.cc:260] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "421ddb0884a04dd8ab35b67c73c44e0a" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43295 } }
I20260809 12:58:34.475358 2852 raft_consensus.cc:388] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:58:34.475526 2852 raft_consensus.cc:482] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:58:34.475723 2852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:58:34.477912 2852 raft_consensus.cc:504] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "421ddb0884a04dd8ab35b67c73c44e0a" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43295 } }
I20260809 12:58:34.478493 2852 leader_election.cc:302] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 421ddb0884a04dd8ab35b67c73c44e0a; no voters:
I20260809 12:58:34.479117 2852 leader_election.cc:288] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:58:34.479364 2857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:58:34.480194 2857 raft_consensus.cc:686] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a [term 1 LEADER]: Becoming Leader. State: Replica: 421ddb0884a04dd8ab35b67c73c44e0a, State: Running, Role: LEADER
I20260809 12:58:34.480927 2857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "421ddb0884a04dd8ab35b67c73c44e0a" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43295 } }
I20260809 12:58:34.481225 2852 sys_catalog.cc:564] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:58:34.484611 2858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "421ddb0884a04dd8ab35b67c73c44e0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "421ddb0884a04dd8ab35b67c73c44e0a" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43295 } } }
I20260809 12:58:34.484578 2859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 421ddb0884a04dd8ab35b67c73c44e0a. Latest consensus state: current_term: 1 leader_uuid: "421ddb0884a04dd8ab35b67c73c44e0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "421ddb0884a04dd8ab35b67c73c44e0a" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43295 } } }
I20260809 12:58:34.485065 2859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a [sys.catalog]: This master's current role is: LEADER
I20260809 12:58:34.485040 2858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a [sys.catalog]: This master's current role is: LEADER
I20260809 12:58:34.486869 2870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:58:34.492858 2870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:58:34.494204 2852 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 12:58:34.515112 2870 catalog_manager.cc:1347] Generated new cluster ID: 6ae94b8be1b144f8ba5824ffd208dc86
I20260809 12:58:34.515282 2870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:58:34.527087 2870 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:58:34.529064 2870 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:58:34.549741 2870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a: Generated new TSK 0
I20260809 12:58:34.550850 2870 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
I20260809 12:58:35.430394 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.235.84:54527/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:43295
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:58:35 dist-test-slave-gp6n krb5kdc[2267](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786280315, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 12:58:35.908684 2907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:58:35.909082 2907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:58:35.909245 2907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:58:35.920841 2907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:58:35.921015 2907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:58:35.921271 2907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:58:35.934120 2907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.235.84:54527/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:43295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:58:35.935895 2907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:58:35.937302 2907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:58:35.958879 2919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:58:35.959539 2920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:58:35.960829 2922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:58:35.961298 2907 server_base.cc:1034] running on GCE node
I20260809 12:58:35.961963 2907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:58:35.963064 2907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:58:35.964329 2907 hybrid_clock.cc:648] HybridClock initialized: now 1786280315964245 us; error 65 us; skew 500 ppm
I20260809 12:58:35.964668 2907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:58:35 dist-test-slave-gp6n krb5kdc[2267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280315, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:58:35.968827 2907 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 12:58:35.970952 2907 webserver.cc:460] Webserver started at http://127.21.235.65:41699/ using document root <none> and password file <none>
I20260809 12:58:35.971984 2907 fs_manager.cc:359] Metadata directory not provided
I20260809 12:58:35.972144 2907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:58:35.972553 2907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:58:35 dist-test-slave-gp6n krb5kdc[2267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280315, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
I20260809 12:58:36.004016 2907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "8dd94b40233e43289a2350f16e10237e"
format_stamp: "Formatted at 2026-08-09 12:58:35 on dist-test-slave-gp6n"
server_key: "74c4faa39c466c4adb35eb7cd28263dd"
server_key_iv: "54df0e4c9b7dfeb28e8af6cdcc2ac0a2"
server_key_version: "kuduclusterkey@0"
I20260809 12:58:36.005888 2907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "8dd94b40233e43289a2350f16e10237e"
format_stamp: "Formatted at 2026-08-09 12:58:35 on dist-test-slave-gp6n"
server_key: "74c4faa39c466c4adb35eb7cd28263dd"
server_key_iv: "54df0e4c9b7dfeb28e8af6cdcc2ac0a2"
server_key_version: "kuduclusterkey@0"
I20260809 12:58:36.020272 2907 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.005s sys 0.007s
I20260809 12:58:36.051146 2929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:58:36.054891 2907 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.004s sys 0.001s
I20260809 12:58:36.055203 2907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "8dd94b40233e43289a2350f16e10237e"
format_stamp: "Formatted at 2026-08-09 12:58:35 on dist-test-slave-gp6n"
server_key: "74c4faa39c466c4adb35eb7cd28263dd"
server_key_iv: "54df0e4c9b7dfeb28e8af6cdcc2ac0a2"
server_key_version: "kuduclusterkey@0"
I20260809 12:58:36.055617 2907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:58:36.109182 2907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:58:36.115233 2907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:58:36.115940 2907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:58:36.122581 2907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:58:36.131763 2907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:58:36.132028 2907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 12:58:36.132345 2907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:58:36.132540 2907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:58:36 dist-test-slave-gp6n krb5kdc[2267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280315, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
Aug 09 12:58:36 dist-test-slave-gp6n krb5kdc[2267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280315, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
I20260809 12:58:36.843170 2907 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:34089
I20260809 12:58:36.845796 2907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:58:36.845499 3055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:34089 every 8 connection(s)
I20260809 12:58:36.848011 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 2907
I20260809 12:58:36.848219 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:58:36.849323 22445 external_mini_cluster.cc:1428] Setting key 5eeed089b66c4660f11fc156f8a849f7
WARNING: no policy specified for kudu/127.21.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.66@KRBTEST.COM" created.
Aug 09 12:58:36 dist-test-slave-gp6n krb5kdc[2267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280315, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:58:36.915671 3058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:36.858983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:41597 (local address 127.21.235.126:43295)
0809 12:58:36.877711 (+ 18728us) server_negotiation.cc:206] Beginning negotiation
0809 12:58:36.877720 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:58:36.877751 (+ 31us) server_negotiation.cc:407] Connection header received
0809 12:58:36.877856 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:58:36.877865 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:58:36.877982 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:58:36.878123 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:58:36.881905 (+ 3782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:36.883761 (+ 1856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:36.887073 (+ 3312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:36.888019 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:36.888132 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:36.905461 (+ 17329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:58:36.905471 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:58:36.905475 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:58:36.905528 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:58:36.910026 (+ 4498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:58:36.911574 (+ 1548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:58:36.911580 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:58:36.911585 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:58:36.911818 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:58:36.912623 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:58:36.912629 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:58:36.912633 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:58:36.913301 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:58:36.913434 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 12:58:36.914777 (+ 1343us) server_negotiation.cc:299] Negotiation successful
0809 12:58:36.915064 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18543,"thread_start_us":18364,"threads_started":1}
I20260809 12:58:36.919021 3059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:36.856728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:43295 (local address 127.21.235.65:41597)
0809 12:58:36.863573 (+ 6845us) negotiation.cc:107] Waiting for socket to connect
0809 12:58:36.863641 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 12:58:36.869351 (+ 5710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:58:36.879956 (+ 10605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:58:36.879970 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:58:36.880694 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:58:36.881578 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:58:36.881601 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:58:36.883998 (+ 2397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:58:36.884003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:58:36.886887 (+ 2884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:58:36.886905 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:58:36.888662 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:58:36.888671 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:58:36.889014 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:36.894591 (+ 5577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:58:36.894712 (+ 121us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:58:36.905056 (+ 10344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:58:36.910377 (+ 5321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:58:36.910388 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:58:36.910431 (+ 43us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:58:36.911334 (+ 903us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:58:36.912004 (+ 670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:58:36.912013 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:58:36.912019 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:58:36.912457 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:58:36.913554 (+ 1097us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:58:36.913564 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:58:36.914201 (+ 637us) client_negotiation.cc:769] Sending connection context
0809 12:58:36.916374 (+ 2173us) client_negotiation.cc:240] Negotiation successful
0809 12:58:36.917037 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6204,"thread_start_us":3944,"threads_started":1}
I20260809 12:58:36.922006 3056 heartbeater.cc:347] Connected to a master server at 127.21.235.126:43295
I20260809 12:58:36.922497 3056 heartbeater.cc:464] Registering TS with master...
I20260809 12:58:36.924090 3056 heartbeater.cc:511] Master 127.21.235.126:43295 requested a full tablet report, sending...
I20260809 12:58:36.927511 2814 ts_manager.cc:194] Registered new tserver with Master: 8dd94b40233e43289a2350f16e10237e (127.21.235.65:34089)
I20260809 12:58:36.930361 2814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:41597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
I20260809 12:58:36.943483 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.235.84:54527/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:43295
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:58:37.649612 3063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:58:37.650029 3063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:58:37.650262 3063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:58:37.659902 3063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:58:37.660112 3063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:58:37.660334 3063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:58:37.673300 3063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.235.84:54527/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:43295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:58:37.674875 3063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:58:37.676597 3063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:58:37.694873 3080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:58:37.694976 3077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:58:37.695170 3078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:58:37.696852 3063 server_base.cc:1034] running on GCE node
I20260809 12:58:37.697875 3063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:58:37.699456 3063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:58:37.700739 3063 hybrid_clock.cc:648] HybridClock initialized: now 1786280317700628 us; error 52 us; skew 500 ppm
I20260809 12:58:37.701160 3063 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:58:37 dist-test-slave-gp6n krb5kdc[2267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280317, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:58:37.706559 3063 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 12:58:37.719007 3063 webserver.cc:460] Webserver started at http://127.21.235.66:40755/ using document root <none> and password file <none>
I20260809 12:58:37.720042 3063 fs_manager.cc:359] Metadata directory not provided
I20260809 12:58:37.720345 3063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:58:37.721087 3063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:58:37 dist-test-slave-gp6n krb5kdc[2267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280317, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
I20260809 12:58:37.746524 3063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "839f4dd26d4e452c8ccacd2d273917d4"
format_stamp: "Formatted at 2026-08-09 12:58:37 on dist-test-slave-gp6n"
server_key: "8f51336b83b382833cf53dd226169738"
server_key_iv: "740b4bc3fd7458353ee57fe7843fa9e5"
server_key_version: "kuduclusterkey@0"
I20260809 12:58:37.748092 3063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "839f4dd26d4e452c8ccacd2d273917d4"
format_stamp: "Formatted at 2026-08-09 12:58:37 on dist-test-slave-gp6n"
server_key: "8f51336b83b382833cf53dd226169738"
server_key_iv: "740b4bc3fd7458353ee57fe7843fa9e5"
server_key_version: "kuduclusterkey@0"
I20260809 12:58:37.759706 3063 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.005s sys 0.005s
I20260809 12:58:37.782372 3087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:58:37.784935 3063 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260809 12:58:37.785141 3063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "839f4dd26d4e452c8ccacd2d273917d4"
format_stamp: "Formatted at 2026-08-09 12:58:37 on dist-test-slave-gp6n"
server_key: "8f51336b83b382833cf53dd226169738"
server_key_iv: "740b4bc3fd7458353ee57fe7843fa9e5"
server_key_version: "kuduclusterkey@0"
I20260809 12:58:37.785501 3063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:58:37.808853 3063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:58:37.813270 3063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:58:37.813561 3063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:58:37.814620 3063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:58:37.816057 3063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:58:37.816136 3063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:58:37.816231 3063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:58:37.816280 3063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:58:37.900027 3063 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:33385
I20260809 12:58:37.900091 3213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:33385 every 8 connection(s)
I20260809 12:58:37.901695 3063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:58:37.908568 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 3063
I20260809 12:58:37.908766 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:58:37.909333 22445 external_mini_cluster.cc:1428] Setting key a57b1941a999a8a916df17f80c3cbd12
Aug 09 12:58:37 dist-test-slave-gp6n krb5kdc[2267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280317, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:58:37.932526 3218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:37.905588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:33011 (local address 127.21.235.126:43295)
0809 12:58:37.906569 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0809 12:58:37.906574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:58:37.907805 (+ 1231us) server_negotiation.cc:407] Connection header received
0809 12:58:37.910514 (+ 2709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:58:37.910521 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:58:37.910625 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:58:37.910932 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:58:37.912691 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:37.914664 (+ 1973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:37.916466 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:37.917576 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:37.917741 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:37.921350 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:58:37.921366 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:58:37.921372 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:58:37.921485 (+ 113us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:58:37.927886 (+ 6401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:58:37.928792 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:58:37.928801 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:58:37.928806 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:58:37.929059 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:58:37.929668 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:58:37.929675 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:58:37.929681 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:58:37.930221 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:58:37.930427 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 12:58:37.931398 (+ 971us) server_negotiation.cc:299] Negotiation successful
0809 12:58:37.931732 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":576,"threads_started":1}
I20260809 12:58:37.933092 3219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:37.906556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:43295 (local address 127.21.235.66:33011)
0809 12:58:37.907487 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0809 12:58:37.907538 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 12:58:37.910099 (+ 2561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:58:37.911188 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:58:37.911197 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:58:37.911802 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:58:37.912494 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:58:37.912515 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:58:37.914858 (+ 2343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:58:37.914862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:58:37.916248 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:58:37.916261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:58:37.917835 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:58:37.917841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:58:37.918058 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:37.918845 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:58:37.918868 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:58:37.921025 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:58:37.928076 (+ 7051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:58:37.928084 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:58:37.928100 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:58:37.928643 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:58:37.929212 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:58:37.929216 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:58:37.929219 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:58:37.929530 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:58:37.930491 (+ 961us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:58:37.930498 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:58:37.930883 (+ 385us) client_negotiation.cc:769] Sending connection context
0809 12:58:37.931191 (+ 308us) client_negotiation.cc:240] Negotiation successful
0809 12:58:37.931698 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":421,"threads_started":1}
I20260809 12:58:37.934978 3056 heartbeater.cc:503] Master 127.21.235.126:43295 was elected leader, sending a full tablet report...
I20260809 12:58:37.935668 3215 heartbeater.cc:347] Connected to a master server at 127.21.235.126:43295
I20260809 12:58:37.936079 3215 heartbeater.cc:464] Registering TS with master...
I20260809 12:58:37.937317 3215 heartbeater.cc:511] Master 127.21.235.126:43295 requested a full tablet report, sending...
I20260809 12:58:37.939414 2814 ts_manager.cc:194] Registered new tserver with Master: 839f4dd26d4e452c8ccacd2d273917d4 (127.21.235.66:33385)
I20260809 12:58:37.940300 2814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:33011
WARNING: no policy specified for kudu/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
I20260809 12:58:37.994128 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.235.84:54527/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:43295
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:58:38.631738 3223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:58:38.632102 3223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:58:38.632340 3223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:58:38.641639 3223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:58:38.641829 3223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:58:38.642107 3223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:58:38.654481 3223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.235.84:54527/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:43295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:58:38.655964 3223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:58:38.657516 3223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:58:38.676190 3232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:58:38.676896 3229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:58:38.676945 3230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:58:38.678282 3223 server_base.cc:1034] running on GCE node
I20260809 12:58:38.679013 3223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:58:38.680004 3223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:58:38.681245 3223 hybrid_clock.cc:648] HybridClock initialized: now 1786280318681176 us; error 55 us; skew 500 ppm
I20260809 12:58:38.681653 3223 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:58:38 dist-test-slave-gp6n krb5kdc[2267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280318, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:58:38.686573 3223 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 12:58:38.688716 3223 webserver.cc:460] Webserver started at http://127.21.235.67:33551/ using document root <none> and password file <none>
I20260809 12:58:38.689612 3223 fs_manager.cc:359] Metadata directory not provided
I20260809 12:58:38.689834 3223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:58:38.690232 3223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:58:38 dist-test-slave-gp6n krb5kdc[2267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280318, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
I20260809 12:58:38.708719 3223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "4e946d2951ef40079947a886e6450765"
format_stamp: "Formatted at 2026-08-09 12:58:38 on dist-test-slave-gp6n"
server_key: "d82aabf03030ad3d195e7388031e2708"
server_key_iv: "6ec2afca59e4a9b5d4285a6630db59dc"
server_key_version: "kuduclusterkey@0"
I20260809 12:58:38.709839 3223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "4e946d2951ef40079947a886e6450765"
format_stamp: "Formatted at 2026-08-09 12:58:38 on dist-test-slave-gp6n"
server_key: "d82aabf03030ad3d195e7388031e2708"
server_key_iv: "6ec2afca59e4a9b5d4285a6630db59dc"
server_key_version: "kuduclusterkey@0"
I20260809 12:58:38.716289 3223 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260809 12:58:38.731717 3241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:58:38.733217 3223 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 12:58:38.733367 3223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "4e946d2951ef40079947a886e6450765"
format_stamp: "Formatted at 2026-08-09 12:58:38 on dist-test-slave-gp6n"
server_key: "d82aabf03030ad3d195e7388031e2708"
server_key_iv: "6ec2afca59e4a9b5d4285a6630db59dc"
server_key_version: "kuduclusterkey@0"
I20260809 12:58:38.733592 3223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:58:38.752005 3223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:58:38.757220 3223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:58:38.757679 3223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:58:38.759225 3223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:58:38.761482 3223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:58:38.761667 3223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:58:38.761909 3223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:58:38.762027 3223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:58:38.845347 3223 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:44173
I20260809 12:58:38.845491 3374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:44173 every 8 connection(s)
I20260809 12:58:38.847146 3223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:58:38.851868 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 3223
I20260809 12:58:38.852067 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:58:38.852583 22445 external_mini_cluster.cc:1428] Setting key f20081da1a1a8717337459a229340d22
Aug 09 12:58:38 dist-test-slave-gp6n krb5kdc[2267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280318, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:58:38.874707 3378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:38.850919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:60355 (local address 127.21.235.126:43295)
0809 12:58:38.851775 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0809 12:58:38.851782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:58:38.853701 (+ 1919us) server_negotiation.cc:407] Connection header received
0809 12:58:38.856591 (+ 2890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:58:38.856595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:58:38.856662 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:58:38.856888 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:58:38.858753 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:38.859758 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:38.861613 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:38.862191 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:38.862315 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:38.867322 (+ 5007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:58:38.867329 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:58:38.867334 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:58:38.867430 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:58:38.871499 (+ 4069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:58:38.872259 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:58:38.872263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:58:38.872265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:58:38.872412 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:58:38.872943 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:58:38.872947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:58:38.872949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:58:38.873276 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:58:38.873385 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 12:58:38.874066 (+ 681us) server_negotiation.cc:299] Negotiation successful
0809 12:58:38.874251 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":487,"threads_started":1}
I20260809 12:58:38.875845 3379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:38.851751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:43295 (local address 127.21.235.67:60355)
0809 12:58:38.853421 (+ 1670us) negotiation.cc:107] Waiting for socket to connect
0809 12:58:38.853469 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 12:58:38.856245 (+ 2776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:58:38.857102 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:58:38.857111 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:58:38.857742 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:58:38.858597 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:58:38.858617 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:58:38.859913 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:58:38.859917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:58:38.861493 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:58:38.861504 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:58:38.862531 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:58:38.862538 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:58:38.862829 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:38.863572 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:58:38.863594 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:58:38.867064 (+ 3470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:58:38.871624 (+ 4560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:58:38.871630 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:58:38.871642 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:58:38.872153 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:58:38.872536 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:58:38.872539 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:58:38.872541 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:58:38.872826 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:58:38.873394 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:58:38.873400 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:58:38.873677 (+ 277us) client_negotiation.cc:769] Sending connection context
0809 12:58:38.873938 (+ 261us) client_negotiation.cc:240] Negotiation successful
0809 12:58:38.874483 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1329,"thread_start_us":962,"threads_started":1}
I20260809 12:58:38.877483 3376 heartbeater.cc:347] Connected to a master server at 127.21.235.126:43295
I20260809 12:58:38.877867 3376 heartbeater.cc:464] Registering TS with master...
I20260809 12:58:38.878954 3376 heartbeater.cc:511] Master 127.21.235.126:43295 requested a full tablet report, sending...
I20260809 12:58:38.881064 2814 ts_manager.cc:194] Registered new tserver with Master: 4e946d2951ef40079947a886e6450765 (127.21.235.67:44173)
I20260809 12:58:38.881888 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:58:38.882261 2814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:60355
I20260809 12:58:38.906906 3378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:38.888813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37318 (local address 127.21.235.126:43295)
0809 12:58:38.888996 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 12:58:38.889041 (+ 45us) server_negotiation.cc:399] Waiting for connection header
0809 12:58:38.890011 (+ 970us) server_negotiation.cc:407] Connection header received
0809 12:58:38.890155 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:58:38.890159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:58:38.890255 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:58:38.890355 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:58:38.892864 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:38.893802 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:38.895882 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:38.896601 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:38.896702 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:38.899876 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:58:38.899889 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:58:38.899892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:58:38.899925 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:58:38.903532 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:58:38.904352 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:58:38.904360 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:58:38.904368 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:58:38.904556 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:58:38.905051 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:58:38.905055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:58:38.905057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:58:38.905400 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:58:38.905758 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0809 12:58:38.906172 (+ 414us) server_negotiation.cc:299] Negotiation successful
0809 12:58:38.906522 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 12:58:38.911720 2814 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:58:38.921905 2814 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:58:38.945039 3215 heartbeater.cc:503] Master 127.21.235.126:43295 was elected leader, sending a full tablet report...
I20260809 12:58:38.954911 3391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:38.939521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53406 (local address 127.21.235.66:33385)
0809 12:58:38.940700 (+ 1179us) server_negotiation.cc:206] Beginning negotiation
0809 12:58:38.940719 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 12:58:38.941904 (+ 1185us) server_negotiation.cc:407] Connection header received
0809 12:58:38.943025 (+ 1121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:58:38.943037 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:58:38.943285 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:58:38.943568 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:58:38.945274 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:38.947313 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:38.949930 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:38.952471 (+ 2541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:38.952674 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:38.953107 (+ 433us) server_negotiation.cc:1035] Waiting for connection context
0809 12:58:38.954045 (+ 938us) server_negotiation.cc:299] Negotiation successful
0809 12:58:38.954270 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":852,"thread_start_us":533,"threads_started":1}
I20260809 12:58:38.954923 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:38.939752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:33385 (local address 127.0.0.1:53406)
0809 12:58:38.941092 (+ 1340us) negotiation.cc:107] Waiting for socket to connect
0809 12:58:38.941127 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 12:58:38.941460 (+ 333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:58:38.943898 (+ 2438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:58:38.943903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:58:38.943927 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:58:38.944890 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:58:38.944921 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:58:38.947482 (+ 2561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:58:38.947486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:58:38.949758 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:58:38.949772 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:58:38.953369 (+ 3597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:58:38.953376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:58:38.953644 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:38.953701 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 12:58:38.953827 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 12:58:38.954104 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1110,"thread_start_us":766,"threads_started":1}
I20260809 12:58:38.961196 3142 tablet_service.cc:1467] Processing CreateTablet for tablet 6c150d65ec0c46498c0787231d3b8c72 (DEFAULT_TABLE table=test-table [id=2b29d6405cdf4844b3b26f5d4a6e978b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:58:38.963553 3142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c150d65ec0c46498c0787231d3b8c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:58:38.965512 3393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:38.941947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:44173 (local address 127.0.0.1:37112)
0809 12:58:38.953088 (+ 11141us) negotiation.cc:107] Waiting for socket to connect
0809 12:58:38.953109 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:58:38.953263 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:58:38.954108 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:58:38.954113 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:58:38.954138 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:58:38.955133 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:58:38.955153 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:58:38.958180 (+ 3027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:58:38.958186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:58:38.961096 (+ 2910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:58:38.961113 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:58:38.964226 (+ 3113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:58:38.964236 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:58:38.964484 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:38.964522 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 12:58:38.964648 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 12:58:38.964787 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10958,"mutex_wait_us":1,"thread_start_us":8494,"threads_started":1}
I20260809 12:58:38.966042 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:38.941800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48302 (local address 127.21.235.65:34089)
0809 12:58:38.943012 (+ 1212us) server_negotiation.cc:206] Beginning negotiation
0809 12:58:38.943020 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:58:38.953528 (+ 10508us) server_negotiation.cc:407] Connection header received
0809 12:58:38.953656 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:58:38.953664 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:58:38.953955 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:58:38.954193 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:58:38.956785 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:38.959139 (+ 2354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:38.961878 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:38.964086 (+ 2208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:38.964308 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:38.964706 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0809 12:58:38.965164 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 12:58:38.965374 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":973,"thread_start_us":733,"threads_started":1}
I20260809 12:58:38.966735 3395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:38.941561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:34089 (local address 127.0.0.1:48302)
0809 12:58:38.952964 (+ 11403us) negotiation.cc:107] Waiting for socket to connect
0809 12:58:38.952998 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 12:58:38.953358 (+ 360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:58:38.954631 (+ 1273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:58:38.954635 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:58:38.954659 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:58:38.956440 (+ 1781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:58:38.956454 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:58:38.959303 (+ 2849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:58:38.959308 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:58:38.961633 (+ 2325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:58:38.961648 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:58:38.964652 (+ 3004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:58:38.964661 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:58:38.964858 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:38.964881 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 12:58:38.965140 (+ 259us) client_negotiation.cc:240] Negotiation successful
0809 12:58:38.965296 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11208,"spinlock_wait_cycles":4992,"thread_start_us":10905,"threads_started":1}
I20260809 12:58:38.967352 3396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:38.942336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37112 (local address 127.21.235.67:44173)
0809 12:58:38.952308 (+ 9972us) server_negotiation.cc:206] Beginning negotiation
0809 12:58:38.952320 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:58:38.953403 (+ 1083us) server_negotiation.cc:407] Connection header received
0809 12:58:38.953529 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:58:38.953538 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:58:38.953858 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:58:38.955177 (+ 1319us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:58:38.956036 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:38.957959 (+ 1923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:38.961304 (+ 3345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:38.963704 (+ 2400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:38.963853 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:38.964344 (+ 491us) server_negotiation.cc:1035] Waiting for connection context
0809 12:58:38.965686 (+ 1342us) server_negotiation.cc:299] Negotiation successful
0809 12:58:38.965896 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9686,"thread_start_us":9440,"threads_started":1}
I20260809 12:58:38.971426 3295 tablet_service.cc:1467] Processing CreateTablet for tablet 6c150d65ec0c46498c0787231d3b8c72 (DEFAULT_TABLE table=test-table [id=2b29d6405cdf4844b3b26f5d4a6e978b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:58:38.971940 2984 tablet_service.cc:1467] Processing CreateTablet for tablet 6c150d65ec0c46498c0787231d3b8c72 (DEFAULT_TABLE table=test-table [id=2b29d6405cdf4844b3b26f5d4a6e978b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:58:38.972958 3295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c150d65ec0c46498c0787231d3b8c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:58:38.974032 2984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c150d65ec0c46498c0787231d3b8c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:58:38.984747 3397 tablet_bootstrap.cc:492] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4: Bootstrap starting.
I20260809 12:58:38.989820 3398 tablet_bootstrap.cc:492] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765: Bootstrap starting.
I20260809 12:58:38.992839 3397 tablet_bootstrap.cc:654] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4: Neither blocks nor log segments found. Creating new log.
I20260809 12:58:38.994764 3399 tablet_bootstrap.cc:492] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e: Bootstrap starting.
I20260809 12:58:38.995533 3397 log.cc:824] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4: Log is configured to *not* fsync() on all Append() calls
I20260809 12:58:38.996466 3398 tablet_bootstrap.cc:654] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765: Neither blocks nor log segments found. Creating new log.
I20260809 12:58:38.998980 3398 log.cc:824] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765: Log is configured to *not* fsync() on all Append() calls
I20260809 12:58:39.000550 3399 tablet_bootstrap.cc:654] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e: Neither blocks nor log segments found. Creating new log.
I20260809 12:58:39.000764 3397 tablet_bootstrap.cc:492] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4: No bootstrap required, opened a new log
I20260809 12:58:39.001277 3397 ts_tablet_manager.cc:1397] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.004s
I20260809 12:58:39.002761 3399 log.cc:824] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e: Log is configured to *not* fsync() on all Append() calls
I20260809 12:58:39.003123 3398 tablet_bootstrap.cc:492] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765: No bootstrap required, opened a new log
I20260809 12:58:39.003510 3398 ts_tablet_manager.cc:1397] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.008s
I20260809 12:58:39.010528 3399 tablet_bootstrap.cc:492] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e: No bootstrap required, opened a new log
I20260809 12:58:39.010903 3399 ts_tablet_manager.cc:1397] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.007s
I20260809 12:58:39.014894 3398 raft_consensus.cc:348] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839f4dd26d4e452c8ccacd2d273917d4" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 33385 } } peers { permanent_uuid: "8dd94b40233e43289a2350f16e10237e" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 34089 } } peers { permanent_uuid: "4e946d2951ef40079947a886e6450765" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44173 } }
I20260809 12:58:39.015415 3398 raft_consensus.cc:374] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:58:39.015594 3398 raft_consensus.cc:729] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e946d2951ef40079947a886e6450765, State: Initialized, Role: FOLLOWER
I20260809 12:58:39.016317 3398 consensus_queue.cc:260] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839f4dd26d4e452c8ccacd2d273917d4" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 33385 } } peers { permanent_uuid: "8dd94b40233e43289a2350f16e10237e" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 34089 } } peers { permanent_uuid: "4e946d2951ef40079947a886e6450765" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44173 } }
I20260809 12:58:39.017444 3397 raft_consensus.cc:348] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839f4dd26d4e452c8ccacd2d273917d4" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 33385 } } peers { permanent_uuid: "8dd94b40233e43289a2350f16e10237e" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 34089 } } peers { permanent_uuid: "4e946d2951ef40079947a886e6450765" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44173 } }
I20260809 12:58:39.018147 3376 heartbeater.cc:503] Master 127.21.235.126:43295 was elected leader, sending a full tablet report...
I20260809 12:58:39.018200 3397 raft_consensus.cc:374] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:58:39.018393 3397 raft_consensus.cc:729] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 839f4dd26d4e452c8ccacd2d273917d4, State: Initialized, Role: FOLLOWER
I20260809 12:58:39.018972 3398 ts_tablet_manager.cc:1428] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260809 12:58:39.019227 3397 consensus_queue.cc:260] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839f4dd26d4e452c8ccacd2d273917d4" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 33385 } } peers { permanent_uuid: "8dd94b40233e43289a2350f16e10237e" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 34089 } } peers { permanent_uuid: "4e946d2951ef40079947a886e6450765" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44173 } }
I20260809 12:58:39.023541 3397 ts_tablet_manager.cc:1428] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4: Time spent starting tablet: real 0.022s user 0.011s sys 0.011s
I20260809 12:58:39.027999 3399 raft_consensus.cc:348] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839f4dd26d4e452c8ccacd2d273917d4" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 33385 } } peers { permanent_uuid: "8dd94b40233e43289a2350f16e10237e" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 34089 } } peers { permanent_uuid: "4e946d2951ef40079947a886e6450765" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44173 } }
I20260809 12:58:39.028457 3399 raft_consensus.cc:374] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:58:39.028581 3399 raft_consensus.cc:729] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dd94b40233e43289a2350f16e10237e, State: Initialized, Role: FOLLOWER
I20260809 12:58:39.029145 3399 consensus_queue.cc:260] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839f4dd26d4e452c8ccacd2d273917d4" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 33385 } } peers { permanent_uuid: "8dd94b40233e43289a2350f16e10237e" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 34089 } } peers { permanent_uuid: "4e946d2951ef40079947a886e6450765" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44173 } }
I20260809 12:58:39.031245 3399 ts_tablet_manager.cc:1428] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e: Time spent starting tablet: real 0.020s user 0.017s sys 0.001s
W20260809 12:58:39.100066 3377 tablet.cc:2369] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:58:39.131779 3057 tablet.cc:2369] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:58:39.155735 3216 tablet.cc:2369] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:58:39.284600 3405 raft_consensus.cc:482] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:58:39.285044 3405 raft_consensus.cc:504] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839f4dd26d4e452c8ccacd2d273917d4" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 33385 } } peers { permanent_uuid: "8dd94b40233e43289a2350f16e10237e" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 34089 } } peers { permanent_uuid: "4e946d2951ef40079947a886e6450765" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44173 } }
I20260809 12:58:39.287644 3405 leader_election.cc:288] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dd94b40233e43289a2350f16e10237e (127.21.235.65:34089), 4e946d2951ef40079947a886e6450765 (127.21.235.67:44173)
I20260809 12:58:39.301097 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:39.288385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:55973 (local address 127.21.235.65:34089)
0809 12:58:39.288667 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 12:58:39.288673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:58:39.291496 (+ 2823us) server_negotiation.cc:407] Connection header received
0809 12:58:39.291810 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:58:39.291816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:58:39.291926 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:58:39.292062 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:58:39.293413 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:39.294809 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:39.298138 (+ 3329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:39.299747 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:39.299923 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:39.299980 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 12:58:39.300675 (+ 695us) server_negotiation.cc:299] Negotiation successful
0809 12:58:39.300772 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:58:39.301653 3407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:39.288765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:34089 (local address 127.21.235.66:55973)
0809 12:58:39.291252 (+ 2487us) negotiation.cc:107] Waiting for socket to connect
0809 12:58:39.291278 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:58:39.291612 (+ 334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:58:39.292079 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:58:39.292103 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:58:39.292139 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:58:39.293230 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:58:39.293243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:58:39.294961 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:58:39.294967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:58:39.297944 (+ 2977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:58:39.297956 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:58:39.300162 (+ 2206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:58:39.300170 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:58:39.300420 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:39.300470 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 12:58:39.300640 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 12:58:39.300830 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2251,"spinlock_wait_cycles":25344,"thread_start_us":1790,"threads_started":1}
I20260809 12:58:39.302105 3396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:39.288509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:44363 (local address 127.21.235.67:44173)
0809 12:58:39.288713 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 12:58:39.288719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:58:39.292468 (+ 3749us) server_negotiation.cc:407] Connection header received
0809 12:58:39.292732 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:58:39.292735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:58:39.292794 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:58:39.292897 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:58:39.294350 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:39.295817 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:39.299126 (+ 3309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:39.300813 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:39.301035 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:39.301139 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 12:58:39.301707 (+ 568us) server_negotiation.cc:299] Negotiation successful
0809 12:58:39.301856 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 12:58:39.302309 3408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:39.289253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:44173 (local address 127.21.235.66:44363)
0809 12:58:39.292211 (+ 2958us) negotiation.cc:107] Waiting for socket to connect
0809 12:58:39.292231 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:58:39.292597 (+ 366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:58:39.292914 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:58:39.292919 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:58:39.292941 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:58:39.294081 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:58:39.294098 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:58:39.295986 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:58:39.295990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:58:39.298920 (+ 2930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:58:39.298935 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:58:39.301250 (+ 2315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:58:39.301259 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:58:39.301527 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:39.301553 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 12:58:39.301634 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 12:58:39.301815 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2682,"thread_start_us":2345,"threads_started":1}
I20260809 12:58:39.303819 3318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c150d65ec0c46498c0787231d3b8c72" candidate_uuid: "839f4dd26d4e452c8ccacd2d273917d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e946d2951ef40079947a886e6450765" is_pre_election: true
I20260809 12:58:39.303751 3004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c150d65ec0c46498c0787231d3b8c72" candidate_uuid: "839f4dd26d4e452c8ccacd2d273917d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dd94b40233e43289a2350f16e10237e" is_pre_election: true
I20260809 12:58:39.304497 3318 raft_consensus.cc:2454] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 839f4dd26d4e452c8ccacd2d273917d4 in term 0.
I20260809 12:58:39.304488 3004 raft_consensus.cc:2454] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 839f4dd26d4e452c8ccacd2d273917d4 in term 0.
I20260809 12:58:39.305470 3088 leader_election.cc:302] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e946d2951ef40079947a886e6450765, 839f4dd26d4e452c8ccacd2d273917d4; no voters:
I20260809 12:58:39.306083 3405 raft_consensus.cc:2781] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:58:39.306278 3405 raft_consensus.cc:482] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:58:39.306447 3405 raft_consensus.cc:3037] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:58:39.310174 3405 raft_consensus.cc:504] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839f4dd26d4e452c8ccacd2d273917d4" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 33385 } } peers { permanent_uuid: "8dd94b40233e43289a2350f16e10237e" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 34089 } } peers { permanent_uuid: "4e946d2951ef40079947a886e6450765" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44173 } }
I20260809 12:58:39.311026 3405 leader_election.cc:288] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [CANDIDATE]: Term 1 election: Requested vote from peers 8dd94b40233e43289a2350f16e10237e (127.21.235.65:34089), 4e946d2951ef40079947a886e6450765 (127.21.235.67:44173)
I20260809 12:58:39.311537 3004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c150d65ec0c46498c0787231d3b8c72" candidate_uuid: "839f4dd26d4e452c8ccacd2d273917d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dd94b40233e43289a2350f16e10237e"
I20260809 12:58:39.311805 3318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c150d65ec0c46498c0787231d3b8c72" candidate_uuid: "839f4dd26d4e452c8ccacd2d273917d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e946d2951ef40079947a886e6450765"
I20260809 12:58:39.311920 3004 raft_consensus.cc:3037] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:58:39.312122 3318 raft_consensus.cc:3037] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:58:39.317497 3004 raft_consensus.cc:2454] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 839f4dd26d4e452c8ccacd2d273917d4 in term 1.
I20260809 12:58:39.317497 3318 raft_consensus.cc:2454] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 839f4dd26d4e452c8ccacd2d273917d4 in term 1.
I20260809 12:58:39.318039 3088 leader_election.cc:302] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e946d2951ef40079947a886e6450765, 839f4dd26d4e452c8ccacd2d273917d4; no voters:
I20260809 12:58:39.318508 3405 raft_consensus.cc:2781] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:58:39.319530 3405 raft_consensus.cc:686] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [term 1 LEADER]: Becoming Leader. State: Replica: 839f4dd26d4e452c8ccacd2d273917d4, State: Running, Role: LEADER
I20260809 12:58:39.320333 3405 consensus_queue.cc:237] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839f4dd26d4e452c8ccacd2d273917d4" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 33385 } } peers { permanent_uuid: "8dd94b40233e43289a2350f16e10237e" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 34089 } } peers { permanent_uuid: "4e946d2951ef40079947a886e6450765" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44173 } }
I20260809 12:58:39.327234 2813 catalog_manager.cc:5495] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 839f4dd26d4e452c8ccacd2d273917d4 (127.21.235.66). New cstate: current_term: 1 leader_uuid: "839f4dd26d4e452c8ccacd2d273917d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "839f4dd26d4e452c8ccacd2d273917d4" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 33385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8dd94b40233e43289a2350f16e10237e" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e946d2951ef40079947a886e6450765" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44173 } health_report { overall_health: UNKNOWN } } }
I20260809 12:58:39.413268 3391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:39.406130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53416 (local address 127.21.235.66:33385)
0809 12:58:39.406319 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 12:58:39.406324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:58:39.406532 (+ 208us) server_negotiation.cc:407] Connection header received
0809 12:58:39.407019 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:58:39.407023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:58:39.407105 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:58:39.407234 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:58:39.407842 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:39.408883 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:39.410747 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:39.411398 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:39.411532 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:39.412107 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:58:39.412722 (+ 615us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:58:39.412884 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 12:58:39.412924 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 12:58:39.412992 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 12:58:39.423959 3318 raft_consensus.cc:1261] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765 [term 1 FOLLOWER]: Refusing update from remote peer 839f4dd26d4e452c8ccacd2d273917d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:58:39.424301 3004 raft_consensus.cc:1261] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e [term 1 FOLLOWER]: Refusing update from remote peer 839f4dd26d4e452c8ccacd2d273917d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:58:39.425158 3409 consensus_queue.cc:1035] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e946d2951ef40079947a886e6450765" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 12:58:39.425868 3405 consensus_queue.cc:1035] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dd94b40233e43289a2350f16e10237e" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:58:39.446780 3416 mvcc.cc:205] Tried to move back new op lower bound from 7316604188349411328 to 7316604187940552704. Current Snapshot: MvccSnapshot[applied={T|T < 7316604188349411328}]
I20260809 12:58:39.479576 2813 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:58:39.480055 2813 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:58:39.485002 2813 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 421ddb0884a04dd8ab35b67c73c44e0a: Sending DeleteTablet for 3 replicas of tablet 6c150d65ec0c46498c0787231d3b8c72
I20260809 12:58:39.486783 2984 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c150d65ec0c46498c0787231d3b8c72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:58:39 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:48302
I20260809 12:58:39.486840 3295 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c150d65ec0c46498c0787231d3b8c72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:58:39 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:37112
I20260809 12:58:39.487938 3142 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c150d65ec0c46498c0787231d3b8c72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:58:39 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:53406
I20260809 12:58:39.488173 3425 tablet_replica.cc:330] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e: stopping tablet replica
I20260809 12:58:39.488714 3425 raft_consensus.cc:2229] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:58:39.489696 3425 raft_consensus.cc:2258] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:58:39.492990 3425 ts_tablet_manager.cc:1905] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:58:39.494621 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 2907
I20260809 12:58:39.500716 3427 tablet_replica.cc:330] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4: stopping tablet replica
I20260809 12:58:39.501328 3427 raft_consensus.cc:2229] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:58:39.502233 3427 raft_consensus.cc:2258] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:58:39.504372 3427 ts_tablet_manager.cc:1905] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:58:39.504518 3425 ts_tablet_manager.cc:1918] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:58:39.504738 3425 log.cc:1196] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/6c150d65ec0c46498c0787231d3b8c72
I20260809 12:58:39.505510 3425 ts_tablet_manager.cc:1939] T 6c150d65ec0c46498c0787231d3b8c72 P 8dd94b40233e43289a2350f16e10237e: Deleting consensus metadata
I20260809 12:58:39.507941 2798 catalog_manager.cc:4841] TS 8dd94b40233e43289a2350f16e10237e (127.21.235.65:34089): tablet 6c150d65ec0c46498c0787231d3b8c72 (table test-table [id=2b29d6405cdf4844b3b26f5d4a6e978b]) successfully deleted
I20260809 12:58:39.512902 3427 ts_tablet_manager.cc:1918] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:58:39.513185 3427 log.cc:1196] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/6c150d65ec0c46498c0787231d3b8c72
I20260809 12:58:39.513237 3426 tablet_replica.cc:330] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765: stopping tablet replica
I20260809 12:58:39.513887 3426 raft_consensus.cc:2229] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:58:39.513978 3427 ts_tablet_manager.cc:1939] T 6c150d65ec0c46498c0787231d3b8c72 P 839f4dd26d4e452c8ccacd2d273917d4: Deleting consensus metadata
I20260809 12:58:39.514544 3426 raft_consensus.cc:2258] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 09 12:58:39 dist-test-slave-gp6n krb5kdc[2267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 12:58:39.516075 2798 catalog_manager.cc:4841] TS 839f4dd26d4e452c8ccacd2d273917d4 (127.21.235.66:33385): tablet 6c150d65ec0c46498c0787231d3b8c72 (table test-table [id=2b29d6405cdf4844b3b26f5d4a6e978b]) successfully deleted
I20260809 12:58:39.516501 3426 ts_tablet_manager.cc:1905] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:58:39.525925 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:39.496639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48312 (local address 127.21.235.65:34089)
0809 12:58:39.496840 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 12:58:39.496847 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:58:39.502094 (+ 5247us) server_negotiation.cc:407] Connection header received
0809 12:58:39.502318 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:58:39.502324 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:58:39.502428 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:58:39.502549 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:58:39.506266 (+ 3717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:39.507547 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:39.509612 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:39.510315 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:39.510508 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:39.517228 (+ 6720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:58:39.517243 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:58:39.517257 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:58:39.517303 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:58:39.521626 (+ 4323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:58:39.522603 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:58:39.522611 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:58:39.522614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:58:39.522757 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:58:39.523219 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:58:39.523224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:58:39.523226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:58:39.523805 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:58:39.524334 (+ 529us) server_negotiation.cc:1035] Waiting for connection context
0809 12:58:39.524949 (+ 615us) server_negotiation.cc:299] Negotiation successful
0809 12:58:39.525269 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 12:58:39.526350 3426 ts_tablet_manager.cc:1918] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:58:39.526669 3426 log.cc:1196] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/6c150d65ec0c46498c0787231d3b8c72
I20260809 12:58:39.527081 3051 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:58:39.527496 3426 ts_tablet_manager.cc:1939] T 6c150d65ec0c46498c0787231d3b8c72 P 4e946d2951ef40079947a886e6450765: Deleting consensus metadata
I20260809 12:58:39.530566 2796 catalog_manager.cc:4841] TS 4e946d2951ef40079947a886e6450765 (127.21.235.67:44173): tablet 6c150d65ec0c46498c0787231d3b8c72 (table test-table [id=2b29d6405cdf4844b3b26f5d4a6e978b]) successfully deleted
I20260809 12:58:39.621454 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 2907
I20260809 12:58:39.643570 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 3063
Aug 09 12:58:39 dist-test-slave-gp6n krb5kdc[2267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.66@KRBTEST.COM
I20260809 12:58:39.661084 3391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:39.644439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53432 (local address 127.21.235.66:33385)
0809 12:58:39.644614 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 12:58:39.644620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:58:39.644693 (+ 73us) server_negotiation.cc:407] Connection header received
0809 12:58:39.644950 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:58:39.644956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:58:39.645036 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:58:39.645152 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:58:39.646747 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:39.647631 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:39.648981 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:39.649468 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:39.649619 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:39.653981 (+ 4362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:58:39.653995 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:58:39.654010 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:58:39.654053 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:58:39.657652 (+ 3599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:58:39.658596 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:58:39.658617 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:58:39.658620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:58:39.658780 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:58:39.659227 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:58:39.659232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:58:39.659243 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:58:39.659694 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:58:39.660054 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0809 12:58:39.660428 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 12:58:39.660703 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:58:39.661886 3209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:58:39.781627 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 3063
I20260809 12:58:39.802767 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 3223
Aug 09 12:58:39 dist-test-slave-gp6n krb5kdc[2267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.67@KRBTEST.COM
I20260809 12:58:39.819154 3431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:58:39.803581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37116 (local address 127.21.235.67:44173)
0809 12:58:39.804052 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0809 12:58:39.804057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:58:39.804071 (+ 14us) server_negotiation.cc:407] Connection header received
0809 12:58:39.804124 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:58:39.804128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:58:39.804187 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:58:39.804317 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:58:39.805679 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:39.806444 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:39.807862 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:58:39.808251 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:58:39.808402 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:58:39.812949 (+ 4547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:58:39.812963 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:58:39.812979 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:58:39.813014 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:58:39.816111 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:58:39.816867 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:58:39.816875 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:58:39.816877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:58:39.817024 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:58:39.817421 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:58:39.817426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:58:39.817428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:58:39.817774 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:58:39.818020 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 12:58:39.818314 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 12:58:39.818761 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":236,"threads_started":1}
I20260809 12:58:39.819794 3369 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:58:39.912192 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 3223
I20260809 12:58:39.933418 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 2775
I20260809 12:58:39.934795 2846 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:58:40.051935 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 2775
I20260809 12:58:40.085006 22445 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 12:58:40.430622 22445 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 12:58:40.430912 22445 mini_ranger.cc:66] Stopping Ranger...
I20260809 12:58:40.775789 22445 mini_ranger.cc:68] Stopped Ranger
2026-08-09 12:58:40.776 UTC [2300] LOG: received smart shutdown request
2026-08-09 12:58:40.780 UTC [2300] LOG: background worker "logical replication launcher" (PID 2309) exited with exit code 1
2026-08-09 12:58:40.780 UTC [2304] LOG: shutting down
2026-08-09 12:58:40.962 UTC [2300] LOG: database system is shut down
2026-08-09T12:58:40Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53973 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:58:41 dist-test-slave-gp6n krb5kdc[3443](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:58:41 dist-test-slave-gp6n krb5kdc[3443](info): set up 2 sockets
Aug 09 12:58:41 dist-test-slave-gp6n krb5kdc[3443](info): commencing operation
krb5kdc: starting...
W20260809 12:58:42.300699 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:58:42 dist-test-slave-gp6n krb5kdc[3443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:58:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:58:42Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.235.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.235.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.235.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/rangeradmin_127.21.235.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.235.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/rangeradmin_127.21.235.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.235.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.235.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.235.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/rangerlookup_127.21.235.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.235.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/rangerlookup_127.21.235.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.235.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.235.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.235.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/HTTP_127.21.235.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.235.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/HTTP_127.21.235.84@KRBTEST.COM.keytab.
I20260809 12:58:42.687017 22445 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskjg7NUQ/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/postgres -l logfile start
2026-08-09 12:58:45.274 UTC [3479] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 12:58:45.274 UTC [3479] LOG: listening on IPv4 address "127.21.235.84", port 44637
2026-08-09 12:58:45.278 UTC [3479] LOG: listening on Unix socket "/tmp/.s.PGSQL.44637"
2026-08-09 12:58:45.301 UTC [3483] LOG: database system was shut down at 2026-08-09 12:58:43 UTC
2026-08-09 12:58:45.304 UTC [3479] LOG: database system is ready to accept connections
I20260809 12:58:46.335692 22445 mini_postgres.cc:96] Postgres bound to 44637
2026-08-09 12:58:46.357 UTC [3491] FATAL: database "slave" does not exist
127.21.235.84:44637 - accepting connections
I20260809 12:58:46.357785 22445 mini_ranger.cc:161] Starting Ranger...
I20260809 12:58:46.386986 22445 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 12:58:46.899513 22445 mini_ranger.cc:87] Created ranger Postgres database
I20260809 12:58:46.899632 22445 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-admin
2026-08-09 12:58:47,208 [I] DB FLAVOR :POSTGRES
2026-08-09 12:58:47,208 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 12:58:47,209 [I] Checking connection..
2026-08-09 12:58:47,209 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 12:58:47,479 [I] Checking connection passed.
2026-08-09 12:58:47,480 [I] --------- Verifying version history table ---------
2026-08-09 12:58:47,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 12:58:47,740 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 12:58:47,740 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 12:58:47,980 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 12:58:47,980 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 12:58:47,980 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 12:58:48.219 UTC [3558] WARNING: there is no transaction in progress
2026-08-09 12:58:48,230 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 12:58:48,231 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 12:58:48,487 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 12:58:48,488 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 12:58:48,488 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 12:58:48,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 12:58:48,966 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gp6n.c.gcp-upstream.internal','N') ;"
2026-08-09 12:58:49,203 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 12:58:49,468 [I] Table x_portal_user does not exist in database ranger
2026-08-09 12:58:49,469 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 12:58:49,732 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 12:58:49,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 12:58:50,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 12:58:50,287 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 12:58:50,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 12:58:51.277 UTC [3680] WARNING: there is no transaction in progress
2026-08-09 12:58:51.294 UTC [3680] WARNING: there is no transaction in progress
2026-08-09 12:58:51.311 UTC [3680] WARNING: there is no transaction in progress
2026-08-09 12:58:51.327 UTC [3680] WARNING: there is no transaction in progress
2026-08-09 12:58:51.343 UTC [3680] WARNING: there is no transaction in progress
2026-08-09 12:58:51.362 UTC [3680] WARNING: there is no transaction in progress
2026-08-09 12:58:51.419 UTC [3680] WARNING: there is no transaction in progress
2026-08-09 12:58:51.425 UTC [3680] WARNING: there is no transaction in progress
2026-08-09 12:58:51.440 UTC [3680] WARNING: there is no transaction in progress
2026-08-09 12:58:51.450 UTC [3680] WARNING: there is no transaction in progress
2026-08-09 12:58:51.461 UTC [3680] WARNING: there is no transaction in progress
2026-08-09 12:58:51.476 UTC [3680] WARNING: there is no transaction in progress
2026-08-09 12:58:51.485 UTC [3680] WARNING: there is no transaction in progress
2026-08-09 12:58:51.492 UTC [3680] WARNING: there is no transaction in progress
2026-08-09 12:58:51.499 UTC [3680] WARNING: there is no transaction in progress
2026-08-09 12:58:51.506 UTC [3680] WARNING: there is no transaction in progress
2026-08-09 12:58:52,171 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 12:58:52,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 12:58:52,429 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 12:58:52,430 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 12:58:52,672 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 12:58:52,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 12:58:52,925 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 12:58:52,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 12:58:53,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 12:58:53,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gp6n.c.gcp-upstream.internal';"
2026-08-09 12:58:53,658 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 12:58:53,659 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 12:58:53,924 [I] DB_PATCHES have already been applied
I20260809 12:58:53.931160 22445 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-admin:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskjg7NUQ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 12:58:53.931290 22445 mini_ranger.cc:193] Using host: 127.21.235.84
I20260809 12:58:53.951288 22445 mini_ranger.cc:227] Ranger admin URL: http://127.21.235.84:47947
Aug 09, 2026 12:58:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:58:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:58:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 12:58:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 12:58:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 12:58:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 12:58:54 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 12:58:55.255919 22445 mini_ranger.cc:160] Time spent starting Ranger: real 8.898s user 0.000s sys 0.055s
Aug 09, 2026 12:59:16 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 12:59:16 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 09, 2026 12:59:16 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 12:59:16 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 12:59:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 12:59:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 12:59:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 12:59:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 12:59:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 12:59:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 12:59:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 12:59:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 12:59:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 12:59:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 12:59:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 12:59:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 12:59:16 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 12:59:17 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 12:59:17.864650 22445 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.235.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.235.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.235.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/rangerkms_127.21.235.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.235.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/rangerkms_127.21.235.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.235.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.235.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.235.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/HTTP_127.21.235.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.235.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/HTTP_127.21.235.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 12:59:18 dist-test-slave-gp6n krb5kdc[3443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280358, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 12:59:18.072504 22445 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 12:59:18.096325 22445 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 12:59:19.028112 22445 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 12:59:19.028244 22445 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-kms
I20260809 12:59:19.028316 22445 mini_ranger_kms.cc:227] Using postgres at 127.21.235.84:44637
2026-08-09 12:59:19,416 [I] DB FLAVOR :POSTGRES
2026-08-09 12:59:19,416 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 12:59:19,416 [I] Checking connection
2026-08-09 12:59:19,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 12:59:19,664 [I] connection success
2026-08-09 12:59:19,665 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 12:59:19,665 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 12:59:19,665 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 12:59:19,907 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 12:59:19,908 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 12:59:19,908 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 12:59:19,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.235.84:44637/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 12:59:20,185 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 12:59:20.664636 22445 mini_ranger_kms.cc:310] Created kms service
I20260809 12:59:20.839154 22445 mini_ranger_kms.cc:326] Created kudu user
I20260809 12:59:20.889912 22445 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 12:59:20 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280358, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
I20260809 12:59:21.330808 22445 mini_ranger_kms.cc:384] Added ranger policy
I20260809 12:59:21.331208 22445 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-kms:/tmp/dist-test-taskjg7NUQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskjg7NUQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 12:59:21.331286 22445 mini_ranger_kms.cc:240] Using host: 127.21.235.84
I20260809 12:59:21.354075 22445 mini_ranger_kms.cc:276] Ranger KMS PID: 3880
I20260809 12:59:21.354218 22445 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.235.84:56787
Aug 09, 2026 12:59:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:59:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:59:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 12:59:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 12:59:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 12:59:23 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 12:59:23 dist-test-slave-gp6n krb5kdc[3443](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786280363, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.235.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:23.848763 22445 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.776s user 0.053s sys 0.367s
I20260809 12:59:23.849031 22445 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 12:59:23.849231 22445 mini_ranger_kms.cc:401] 127.21.235.84:56787/kms/v1/keys
Aug 09 12:59:24 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280363, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.235.84@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
Aug 09 12:59:24 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280363, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.235.84@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
Aug 09 12:59:26 dist-test-slave-gp6n krb5kdc[3443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 12:59:26.827113 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:40979
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:40979
--ranger_config_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.235.84:56787/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:59:27.188812 3934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:27.189116 3934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:27.189216 3934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:27.197567 3934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:59:27.197686 3934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:27.197737 3934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:27.197804 3934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:59:27.197858 3934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:59:27.209527 3934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.235.84:56787/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:40979
--ranger_config_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:40979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:27.210938 3934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:27.212340 3934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:27.226349 3941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:27.226364 3940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:27.226377 3943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:27.226742 3934 server_base.cc:1034] running on GCE node
I20260809 12:59:27.227843 3934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:27.229362 3934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:27.230613 3934 hybrid_clock.cc:648] HybridClock initialized: now 1786280367230557 us; error 45 us; skew 500 ppm
I20260809 12:59:27.231009 3934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:27 dist-test-slave-gp6n krb5kdc[3443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280367, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:27.238534 3934 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:59:27.240172 3934 webserver.cc:460] Webserver started at http://127.21.235.126:40063/ using document root <none> and password file <none>
I20260809 12:59:27.240882 3934 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:27.241029 3934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:27.241365 3934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:59:27 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280367, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
I20260809 12:59:27.297685 3934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "6738c5c21293491cbd85d67c7f04ecb4"
format_stamp: "Formatted at 2026-08-09 12:59:27 on dist-test-slave-gp6n"
server_key: "c56dcae191c8cd47a50502e67a27e011"
server_key_iv: "458e4550a59f4351a76111f4886cb6b6"
server_key_version: "kuduclusterkey@0"
I20260809 12:59:27.298763 3934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "6738c5c21293491cbd85d67c7f04ecb4"
format_stamp: "Formatted at 2026-08-09 12:59:27 on dist-test-slave-gp6n"
server_key: "c56dcae191c8cd47a50502e67a27e011"
server_key_iv: "458e4550a59f4351a76111f4886cb6b6"
server_key_version: "kuduclusterkey@0"
I20260809 12:59:27.304910 3934 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 12:59:27.323504 3951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:27.325161 3934 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 12:59:27.325394 3934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "6738c5c21293491cbd85d67c7f04ecb4"
format_stamp: "Formatted at 2026-08-09 12:59:27 on dist-test-slave-gp6n"
server_key: "c56dcae191c8cd47a50502e67a27e011"
server_key_iv: "458e4550a59f4351a76111f4886cb6b6"
server_key_version: "kuduclusterkey@0"
I20260809 12:59:27.325671 3934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:27.339378 3934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:27.343840 3934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:27.344185 3934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:27.365121 3934 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:40979
I20260809 12:59:27.365121 4009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:40979 every 8 connection(s)
I20260809 12:59:27.366660 3934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:59:27.370831 4010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:59:27.376426 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 3934
I20260809 12:59:27.376657 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:59:27.377507 22445 external_mini_cluster.cc:1428] Setting key ef47e0cbbbe2e76d8f2f28cc500dca3b
I20260809 12:59:27.384110 4010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4: Bootstrap starting.
I20260809 12:59:27.389504 4010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4: Neither blocks nor log segments found. Creating new log.
I20260809 12:59:27.391321 4010 log.cc:824] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:27.395135 4010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4: No bootstrap required, opened a new log
I20260809 12:59:27.404738 4010 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6738c5c21293491cbd85d67c7f04ecb4" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40979 } }
I20260809 12:59:27.405265 4010 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:59:27.405408 4010 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6738c5c21293491cbd85d67c7f04ecb4, State: Initialized, Role: FOLLOWER
Aug 09 12:59:27 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:59:27.406157 4010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6738c5c21293491cbd85d67c7f04ecb4" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40979 } }
I20260809 12:59:27.406471 4010 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:59:27.406644 4010 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:59:27.406831 4010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:59:27.410557 4010 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6738c5c21293491cbd85d67c7f04ecb4" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40979 } }
I20260809 12:59:27.411185 4010 leader_election.cc:302] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6738c5c21293491cbd85d67c7f04ecb4; no voters:
I20260809 12:59:27.412922 4010 leader_election.cc:288] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:59:27.413056 4015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:59:27.417670 4013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:27.379708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39104 (local address 127.21.235.126:40979)
0809 12:59:27.391035 (+ 11327us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:27.391055 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:27.391534 (+ 479us) server_negotiation.cc:407] Connection header received
0809 12:59:27.394052 (+ 2518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:27.394084 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:27.394643 (+ 559us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:27.395332 (+ 689us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:27.396720 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:27.398940 (+ 2220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:27.400848 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:27.402016 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:27.402236 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:27.407362 (+ 5126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:27.407373 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:27.407395 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:27.407454 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:27.411748 (+ 4294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:27.412735 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:27.412749 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:27.412763 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:27.413001 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:27.413431 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:27.413439 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:27.413445 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:27.414446 (+ 1001us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:27.414872 (+ 426us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:27.415271 (+ 399us) server_negotiation.cc:299] Negotiation successful
0809 12:59:27.415863 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11025,"thread_start_us":10791,"threads_started":1}
I20260809 12:59:27.424664 4010 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:59:27.425048 4015 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [term 1 LEADER]: Becoming Leader. State: Replica: 6738c5c21293491cbd85d67c7f04ecb4, State: Running, Role: LEADER
I20260809 12:59:27.425760 4015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6738c5c21293491cbd85d67c7f04ecb4" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40979 } }
I20260809 12:59:27.428689 4017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6738c5c21293491cbd85d67c7f04ecb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6738c5c21293491cbd85d67c7f04ecb4" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40979 } } }
I20260809 12:59:27.429096 4017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [sys.catalog]: This master's current role is: LEADER
I20260809 12:59:27.429658 4018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6738c5c21293491cbd85d67c7f04ecb4. Latest consensus state: current_term: 1 leader_uuid: "6738c5c21293491cbd85d67c7f04ecb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6738c5c21293491cbd85d67c7f04ecb4" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40979 } } }
I20260809 12:59:27.429931 4018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [sys.catalog]: This master's current role is: LEADER
I20260809 12:59:27.435535 4024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:59:27.442879 4024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:59:27.443676 4010 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 12:59:27.469602 4024 catalog_manager.cc:1347] Generated new cluster ID: e3bea2d629694fdbbf7ecb0bfc6f61db
I20260809 12:59:27.470032 4024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:59:27.485018 4024 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:59:27.486740 4024 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:59:27.504976 4024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4: Generated new TSK 0
I20260809 12:59:27.505896 4024 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
I20260809 12:59:28.339259 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.235.84:56787/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40979
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:59:28 dist-test-slave-gp6n krb5kdc[3443](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786280368, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 12:59:28.774279 4061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:28.774653 4061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:28.774837 4061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:28.783938 4061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:28.784041 4061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:28.784227 4061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:59:28.795047 4061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.235.84:56787/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:28.796531 4061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:28.798039 4061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:28.816156 4072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:28.816749 4074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:28.816931 4071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:28.818872 4061 server_base.cc:1034] running on GCE node
I20260809 12:59:28.819701 4061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:28.821012 4061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:28.822263 4061 hybrid_clock.cc:648] HybridClock initialized: now 1786280368822155 us; error 92 us; skew 500 ppm
I20260809 12:59:28.822690 4061 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:28 dist-test-slave-gp6n krb5kdc[3443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280368, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:28.827129 4061 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 12:59:28.829857 4061 webserver.cc:460] Webserver started at http://127.21.235.65:37407/ using document root <none> and password file <none>
I20260809 12:59:28.831036 4061 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:28.831244 4061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:28.831602 4061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:59:28 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280368, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
I20260809 12:59:28.851570 4061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "71340472ba4044feb19d3aa86f6e22b4"
format_stamp: "Formatted at 2026-08-09 12:59:28 on dist-test-slave-gp6n"
server_key: "89806b258376b09734c3f7ae5f0d33a0"
server_key_iv: "388d4f7c4547a620a96c9fad54bc711f"
server_key_version: "kuduclusterkey@0"
I20260809 12:59:28.852710 4061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "71340472ba4044feb19d3aa86f6e22b4"
format_stamp: "Formatted at 2026-08-09 12:59:28 on dist-test-slave-gp6n"
server_key: "89806b258376b09734c3f7ae5f0d33a0"
server_key_iv: "388d4f7c4547a620a96c9fad54bc711f"
server_key_version: "kuduclusterkey@0"
I20260809 12:59:28.860204 4061 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260809 12:59:28.893810 4081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:28.895943 4061 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.003s sys 0.005s
I20260809 12:59:28.896163 4061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "71340472ba4044feb19d3aa86f6e22b4"
format_stamp: "Formatted at 2026-08-09 12:59:28 on dist-test-slave-gp6n"
server_key: "89806b258376b09734c3f7ae5f0d33a0"
server_key_iv: "388d4f7c4547a620a96c9fad54bc711f"
server_key_version: "kuduclusterkey@0"
I20260809 12:59:28.896466 4061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:28.917482 4061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:28.922878 4061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:28.923226 4061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:28.924422 4061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:59:28.926343 4061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:59:28.926505 4061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260809 12:59:28.926644 4061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:59:28.926712 4061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:29.003576 4061 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:40763
I20260809 12:59:29.003597 4208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:40763 every 8 connection(s)
I20260809 12:59:29.004964 4061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:59:29.009136 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 4061
I20260809 12:59:29.009410 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:59:29.010499 22445 external_mini_cluster.cc:1428] Setting key a3aa410fa95c9abd1ee9dd847527198a
Aug 09 12:59:29 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280368, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:59:29.042323 4212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:29.008317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:39635 (local address 127.21.235.126:40979)
0809 12:59:29.015161 (+ 6844us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:29.015171 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:29.019037 (+ 3866us) server_negotiation.cc:407] Connection header received
0809 12:59:29.021847 (+ 2810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:29.021853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:29.021940 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:29.022052 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:29.024587 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:29.025737 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:29.028355 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:29.029061 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:29.029189 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:29.034223 (+ 5034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:29.034230 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:29.034235 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:29.034289 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:29.038139 (+ 3850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:29.039196 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:29.039202 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:29.039206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:29.039376 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:29.039976 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:29.039985 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:29.039989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:29.040384 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:29.040536 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:29.041606 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0809 12:59:29.041886 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6659,"thread_start_us":9228,"threads_started":1}
I20260809 12:59:29.045159 4213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:29.009175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40979 (local address 127.21.235.65:39635)
0809 12:59:29.018864 (+ 9689us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:29.018907 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:29.020677 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:29.022546 (+ 1869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:29.022559 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:29.023544 (+ 985us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:29.024369 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:29.024394 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:29.026020 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:29.026025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:29.028167 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:29.028183 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:29.029614 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:29.029621 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:29.030032 (+ 411us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:29.030925 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:29.030962 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:29.033926 (+ 2964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:29.038361 (+ 4435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:29.038371 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:29.038390 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:29.039045 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:29.039549 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:29.039553 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:29.039557 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:29.039830 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:29.040634 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:29.040642 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:29.041132 (+ 490us) client_negotiation.cc:769] Sending connection context
0809 12:59:29.042962 (+ 1830us) client_negotiation.cc:240] Negotiation successful
0809 12:59:29.043609 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9412,"thread_start_us":11788,"threads_started":1}
WARNING: no policy specified for kudu/127.21.235.66@KRBTEST.COM; defaulting to no policy
I20260809 12:59:29.047518 4209 heartbeater.cc:347] Connected to a master server at 127.21.235.126:40979
I20260809 12:59:29.047871 4209 heartbeater.cc:464] Registering TS with master...
I20260809 12:59:29.048807 4209 heartbeater.cc:511] Master 127.21.235.126:40979 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.66@KRBTEST.COM" created.
I20260809 12:59:29.051717 3968 ts_manager.cc:194] Registered new tserver with Master: 71340472ba4044feb19d3aa86f6e22b4 (127.21.235.65:40763)
I20260809 12:59:29.053820 3968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:39635
Aug 09 12:59:29 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280368, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
I20260809 12:59:29.114812 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.235.84:56787/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40979
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:59:29 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280368, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
W20260809 12:59:29.628499 4217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:29.629040 4217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:29.629257 4217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:29.641266 4217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:29.641487 4217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:29.641746 4217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:59:29.658316 4217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.235.84:56787/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:29.660429 4217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:29.662257 4217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:29.702628 4225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:29.703718 4226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:29.707419 4230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:29.708405 4217 server_base.cc:1034] running on GCE node
I20260809 12:59:29.709235 4217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:29.710659 4217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:29.711999 4217 hybrid_clock.cc:648] HybridClock initialized: now 1786280369711919 us; error 74 us; skew 500 ppm
I20260809 12:59:29.712401 4217 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:29 dist-test-slave-gp6n krb5kdc[3443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280369, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:29.717602 4217 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 12:59:29.719683 4217 webserver.cc:460] Webserver started at http://127.21.235.66:39425/ using document root <none> and password file <none>
I20260809 12:59:29.720656 4217 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:29.720809 4217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:29.721201 4217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:59:29 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280369, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
I20260809 12:59:29.752511 4217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "c68f5f098a254a9092f92f51fe72e611"
format_stamp: "Formatted at 2026-08-09 12:59:29 on dist-test-slave-gp6n"
server_key: "0bec4f8cdd2c9db758e2e410f1244ed0"
server_key_iv: "9d3518c640a40e17248f17976447a799"
server_key_version: "kuduclusterkey@0"
I20260809 12:59:29.753836 4217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "c68f5f098a254a9092f92f51fe72e611"
format_stamp: "Formatted at 2026-08-09 12:59:29 on dist-test-slave-gp6n"
server_key: "0bec4f8cdd2c9db758e2e410f1244ed0"
server_key_iv: "9d3518c640a40e17248f17976447a799"
server_key_version: "kuduclusterkey@0"
I20260809 12:59:29.764845 4217 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.006s sys 0.003s
I20260809 12:59:29.793012 4245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:29.795074 4217 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.005s sys 0.008s
I20260809 12:59:29.795277 4217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "c68f5f098a254a9092f92f51fe72e611"
format_stamp: "Formatted at 2026-08-09 12:59:29 on dist-test-slave-gp6n"
server_key: "0bec4f8cdd2c9db758e2e410f1244ed0"
server_key_iv: "9d3518c640a40e17248f17976447a799"
server_key_version: "kuduclusterkey@0"
I20260809 12:59:29.795562 4217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:29.824112 4217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:29.828804 4217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:29.829142 4217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:29.830335 4217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:59:29.831900 4217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:59:29.831974 4217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:29.832147 4217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:59:29.832198 4217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:29.872777 4217 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:40993
I20260809 12:59:29.872810 4369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:40993 every 8 connection(s)
I20260809 12:59:29.874122 4217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:59:29.882694 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 4217
I20260809 12:59:29.882874 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:59:29.883493 22445 external_mini_cluster.cc:1428] Setting key 21c665a6f706b79d72c8ce3adb0e64fa
Aug 09 12:59:29 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280369, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:59:29.895902 4372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:29.876991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:41875 (local address 127.21.235.126:40979)
0809 12:59:29.877497 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:29.877502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:29.878817 (+ 1315us) server_negotiation.cc:407] Connection header received
0809 12:59:29.880665 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:29.880671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:29.880771 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:29.880896 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:29.882326 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:29.883396 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:29.885793 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:29.886505 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:29.886658 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:29.890047 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:29.890053 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:29.890060 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:29.890097 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:29.892815 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:29.893504 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:29.893511 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:29.893517 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:29.893652 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:29.894096 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:29.894102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:29.894105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:29.894386 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:29.894612 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:29.895287 (+ 675us) server_negotiation.cc:299] Negotiation successful
0809 12:59:29.895523 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":280,"threads_started":1}
I20260809 12:59:29.896951 4373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:29.877691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40979 (local address 127.21.235.66:41875)
0809 12:59:29.878647 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:29.878685 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:29.880369 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:29.881136 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:29.881143 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:29.881593 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:29.882151 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:29.882168 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:29.883602 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:29.883610 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:29.885594 (+ 1984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:29.885610 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:29.886764 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:29.886772 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:29.887001 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:29.887709 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:29.887732 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:29.889824 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:29.892957 (+ 3133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:29.892966 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:29.892982 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:29.893382 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:29.893772 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:29.893777 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:29.893781 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:29.893981 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:29.894587 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:29.894594 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:29.894924 (+ 330us) client_negotiation.cc:769] Sending connection context
0809 12:59:29.895197 (+ 273us) client_negotiation.cc:240] Negotiation successful
0809 12:59:29.895716 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":399,"threads_started":1}
I20260809 12:59:29.898523 4370 heartbeater.cc:347] Connected to a master server at 127.21.235.126:40979
I20260809 12:59:29.898856 4370 heartbeater.cc:464] Registering TS with master...
I20260809 12:59:29.899699 4370 heartbeater.cc:511] Master 127.21.235.126:40979 requested a full tablet report, sending...
I20260809 12:59:29.901479 3968 ts_manager.cc:194] Registered new tserver with Master: c68f5f098a254a9092f92f51fe72e611 (127.21.235.66:40993)
I20260809 12:59:29.902462 3968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:41875
WARNING: no policy specified for kudu/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
I20260809 12:59:29.957671 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.235.84:56787/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40979
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:59:30.058085 4209 heartbeater.cc:503] Master 127.21.235.126:40979 was elected leader, sending a full tablet report...
W20260809 12:59:30.351567 4376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:30.351831 4376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:30.352021 4376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:30.359207 4376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:30.359318 4376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:30.359431 4376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:59:30.371899 4376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.235.84:56787/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:30.373521 4376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:30.375151 4376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:30.388154 4381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:30.388187 4382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:30.388388 4385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:30.389272 4376 server_base.cc:1034] running on GCE node
I20260809 12:59:30.389874 4376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:30.390888 4376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:30.392153 4376 hybrid_clock.cc:648] HybridClock initialized: now 1786280370392112 us; error 50 us; skew 500 ppm
I20260809 12:59:30.392516 4376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:30 dist-test-slave-gp6n krb5kdc[3443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280370, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:30.396075 4376 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 12:59:30.397696 4376 webserver.cc:460] Webserver started at http://127.21.235.67:45981/ using document root <none> and password file <none>
I20260809 12:59:30.398494 4376 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:30.398604 4376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:30.398874 4376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:59:30 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280370, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
I20260809 12:59:30.414312 4376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "83f898636ee04f7690161e948a6d0033"
format_stamp: "Formatted at 2026-08-09 12:59:30 on dist-test-slave-gp6n"
server_key: "7f9c207f51fc9476f391636ba19005f8"
server_key_iv: "700bd232c3c43b397ba5b2f7bb06694d"
server_key_version: "kuduclusterkey@0"
I20260809 12:59:30.415504 4376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "83f898636ee04f7690161e948a6d0033"
format_stamp: "Formatted at 2026-08-09 12:59:30 on dist-test-slave-gp6n"
server_key: "7f9c207f51fc9476f391636ba19005f8"
server_key_iv: "700bd232c3c43b397ba5b2f7bb06694d"
server_key_version: "kuduclusterkey@0"
I20260809 12:59:30.421505 4376 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260809 12:59:30.444725 4393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:30.446583 4376 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.011s sys 0.000s
I20260809 12:59:30.446791 4376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "83f898636ee04f7690161e948a6d0033"
format_stamp: "Formatted at 2026-08-09 12:59:30 on dist-test-slave-gp6n"
server_key: "7f9c207f51fc9476f391636ba19005f8"
server_key_iv: "700bd232c3c43b397ba5b2f7bb06694d"
server_key_version: "kuduclusterkey@0"
I20260809 12:59:30.447057 4376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:30.461432 4376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:30.465425 4376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:30.465728 4376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:30.466682 4376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:59:30.467996 4376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:59:30.468070 4376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:30.468241 4376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:59:30.468297 4376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:30.501063 4376 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:46441
I20260809 12:59:30.501169 4522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:46441 every 8 connection(s)
I20260809 12:59:30.502512 4376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:59:30.510109 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 4376
I20260809 12:59:30.510330 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:59:30.510871 22445 external_mini_cluster.cc:1428] Setting key 55b60a557bd6be5cd9bb49418bba2fd2
Aug 09 12:59:30 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280370, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:59:30.522799 4525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:30.505234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:35847 (local address 127.21.235.126:40979)
0809 12:59:30.505727 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:30.505732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:30.506781 (+ 1049us) server_negotiation.cc:407] Connection header received
0809 12:59:30.508570 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:30.508574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:30.508662 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:30.508762 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:30.510474 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:30.511244 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:30.512972 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:30.513507 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:30.513607 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:30.517096 (+ 3489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:30.517102 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:30.517105 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:30.517148 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:30.519986 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:30.520707 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:30.520711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:30.520713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:30.520839 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:30.521254 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:30.521257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:30.521259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:30.521517 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:30.521629 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:30.522270 (+ 641us) server_negotiation.cc:299] Negotiation successful
0809 12:59:30.522537 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":268,"threads_started":1}
I20260809 12:59:30.523712 4526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:30.505839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40979 (local address 127.21.235.67:35847)
0809 12:59:30.506620 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:30.506668 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:30.508315 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:30.508970 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:30.508978 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:30.509396 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:30.510156 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:30.510184 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:30.511406 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:30.511410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:30.512843 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:30.512854 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:30.513773 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:30.513781 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:30.514073 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:30.514802 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:30.514824 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:30.516899 (+ 2075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:30.520113 (+ 3214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:30.520119 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:30.520130 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:30.520589 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:30.520950 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:30.520953 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:30.520956 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:30.521161 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:30.521622 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:30.521627 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:30.521924 (+ 297us) client_negotiation.cc:769] Sending connection context
0809 12:59:30.522149 (+ 225us) client_negotiation.cc:240] Negotiation successful
0809 12:59:30.522601 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":269,"threads_started":1}
I20260809 12:59:30.525233 4523 heartbeater.cc:347] Connected to a master server at 127.21.235.126:40979
I20260809 12:59:30.525522 4523 heartbeater.cc:464] Registering TS with master...
I20260809 12:59:30.526453 4523 heartbeater.cc:511] Master 127.21.235.126:40979 requested a full tablet report, sending...
I20260809 12:59:30.527805 3968 ts_manager.cc:194] Registered new tserver with Master: 83f898636ee04f7690161e948a6d0033 (127.21.235.67:46441)
I20260809 12:59:30.528654 3968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:35847
I20260809 12:59:30.538949 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:59:30.540702 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 4061
Aug 09 12:59:30 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 12:59:30.557785 4528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:30.541596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40244 (local address 127.21.235.65:40763)
0809 12:59:30.542199 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:30.542206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:30.542231 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:59:30.542288 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:30.542294 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:30.542574 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:30.542702 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:30.544225 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:30.545244 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:30.546824 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:30.547397 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:30.547504 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:30.552016 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:30.552029 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:30.552042 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:30.552097 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:30.555270 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:30.555958 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:30.555965 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:30.555967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:30.556100 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:30.556461 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:30.556464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:30.556466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:30.556809 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:30.557042 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:30.557286 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 12:59:30.557513 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":305,"threads_started":1}
I20260809 12:59:30.559019 4204 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:30.643311 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 4061
I20260809 12:59:30.663623 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 4217
Aug 09 12:59:30 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.66@KRBTEST.COM
I20260809 12:59:30.681087 4530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:30.664589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38046 (local address 127.21.235.66:40993)
0809 12:59:30.665208 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:30.665216 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:30.665243 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:59:30.665368 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:30.665375 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:30.665600 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:30.665739 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:30.667094 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:30.668149 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:30.669497 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:30.670065 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:30.670165 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:30.675021 (+ 4856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:30.675033 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:30.675047 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:30.675083 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:30.677955 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:30.678920 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:30.678927 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:30.678929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:30.679081 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:30.679464 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:30.679467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:30.679469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:30.679849 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:30.680105 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:30.680556 (+ 451us) server_negotiation.cc:299] Negotiation successful
0809 12:59:30.680774 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":352,"threads_started":1}
I20260809 12:59:30.682549 4365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:30.765775 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 4217
I20260809 12:59:30.791035 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 4376
Aug 09 12:59:30 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.67@KRBTEST.COM
I20260809 12:59:30.808877 4532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:30.792218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33690 (local address 127.21.235.67:46441)
0809 12:59:30.792887 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:30.792894 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:30.792922 (+ 28us) server_negotiation.cc:407] Connection header received
0809 12:59:30.792982 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:30.792989 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:30.793208 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:30.793394 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:30.794925 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:30.795984 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:30.797258 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:30.797823 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:30.797922 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:30.802618 (+ 4696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:30.802630 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:30.802646 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:30.802688 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:30.805932 (+ 3244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:30.806663 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:30.806671 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:30.806674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:30.806841 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:30.807214 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:30.807218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:30.807220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:30.807600 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:30.807860 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:30.808227 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 12:59:30.808433 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":354,"threads_started":1}
I20260809 12:59:30.810189 4516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:30.900060 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 4376
I20260809 12:59:30.919232 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 3934
I20260809 12:59:30.920693 4003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:31.032218 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 3934
I20260809 12:59:31.061641 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:40979
--webserver_interface=127.21.235.126
--webserver_port=40063
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:40979
--ranger_config_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.235.84:56787/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.235.84:56787/kms with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:59:31.409168 4535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:31.409564 4535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:31.409724 4535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:31.417266 4535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:59:31.417428 4535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:31.417541 4535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:31.417644 4535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:59:31.417730 4535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:59:31.427965 4535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.235.84:56787/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:40979
--ranger_config_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:40979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=40063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:31.429217 4535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:31.430616 4535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:31.442966 4541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:31.442992 4544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:31.442907 4540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:31.443717 4535 server_base.cc:1034] running on GCE node
I20260809 12:59:31.444478 4535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:31.445842 4535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:31.447126 4535 hybrid_clock.cc:648] HybridClock initialized: now 1786280371447068 us; error 54 us; skew 500 ppm
I20260809 12:59:31.447548 4535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:31 dist-test-slave-gp6n krb5kdc[3443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280371, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:31.450917 4535 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:59:31.452968 4535 webserver.cc:460] Webserver started at http://127.21.235.126:40063/ using document root <none> and password file <none>
I20260809 12:59:31.453624 4535 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:31.453778 4535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 12:59:31 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280371, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
I20260809 12:59:31.506649 4535 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:59:31.509891 4553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:31.511561 4535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:59:31.511786 4535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "6738c5c21293491cbd85d67c7f04ecb4"
format_stamp: "Formatted at 2026-08-09 12:59:27 on dist-test-slave-gp6n"
server_key: "c56dcae191c8cd47a50502e67a27e011"
server_key_iv: "458e4550a59f4351a76111f4886cb6b6"
server_key_version: "kuduclusterkey@0"
I20260809 12:59:31.512782 4535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:31.525178 4535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:31.528824 4535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:31.529121 4535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:31.549038 4535 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:40979
I20260809 12:59:31.549062 4610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:40979 every 8 connection(s)
I20260809 12:59:31.550642 4535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:59:31.559171 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 4535
I20260809 12:59:31.560264 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.235.84:56787/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:40763
--local_ip_for_outbound_sockets=127.21.235.65
--tserver_master_addrs=127.21.235.126:40979
--webserver_port=37407
--webserver_interface=127.21.235.65
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.235.84:56787/kms with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:59:31.562098 4611 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:59:31.569753 4611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4: Bootstrap starting.
I20260809 12:59:31.590075 4611 log.cc:824] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:31.599339 4611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:59:31.599869 4611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4: Bootstrap complete.
I20260809 12:59:31.609112 4611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6738c5c21293491cbd85d67c7f04ecb4" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40979 } }
I20260809 12:59:31.610006 4611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6738c5c21293491cbd85d67c7f04ecb4, State: Initialized, Role: FOLLOWER
I20260809 12:59:31.610658 4611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6738c5c21293491cbd85d67c7f04ecb4" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40979 } }
I20260809 12:59:31.610901 4611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:59:31.611037 4611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:59:31.611227 4611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [term 1 FOLLOWER]: Advancing to term 2
I20260809 12:59:31.615088 4611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6738c5c21293491cbd85d67c7f04ecb4" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40979 } }
I20260809 12:59:31.615690 4611 leader_election.cc:302] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6738c5c21293491cbd85d67c7f04ecb4; no voters:
I20260809 12:59:31.616390 4611 leader_election.cc:288] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 12:59:31.616549 4616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 12:59:31.618247 4616 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [term 2 LEADER]: Becoming Leader. State: Replica: 6738c5c21293491cbd85d67c7f04ecb4, State: Running, Role: LEADER
I20260809 12:59:31.618968 4611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:59:31.618805 4616 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6738c5c21293491cbd85d67c7f04ecb4" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40979 } }
I20260809 12:59:31.621524 4617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6738c5c21293491cbd85d67c7f04ecb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6738c5c21293491cbd85d67c7f04ecb4" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40979 } } }
I20260809 12:59:31.621982 4617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [sys.catalog]: This master's current role is: LEADER
I20260809 12:59:31.622228 4618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6738c5c21293491cbd85d67c7f04ecb4. Latest consensus state: current_term: 2 leader_uuid: "6738c5c21293491cbd85d67c7f04ecb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6738c5c21293491cbd85d67c7f04ecb4" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40979 } } }
I20260809 12:59:31.622772 4618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4 [sys.catalog]: This master's current role is: LEADER
I20260809 12:59:31.629958 4621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:59:31.636795 4621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:59:31.638854 4621 catalog_manager.cc:1259] Loaded cluster ID: e3bea2d629694fdbbf7ecb0bfc6f61db
I20260809 12:59:31.639048 4621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:59:31.644414 4621 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:59:31.645944 4621 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4: Loaded TSK: 0
I20260809 12:59:31.647593 4621 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:59:31.655551 4611 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260809 12:59:32.024099 4614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:32.024509 4614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:32.024677 4614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:32.036204 4614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:32.036352 4614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:32.036569 4614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:59:32.048875 4614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.235.84:56787/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:40763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=37407
--tserver_master_addrs=127.21.235.126:40979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:32.050266 4614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:32.051643 4614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:32.085130 4652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:32.085085 4653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:32.087548 4655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:32.092414 4614 server_base.cc:1034] running on GCE node
I20260809 12:59:32.093178 4614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:32.094393 4614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:32.095690 4614 hybrid_clock.cc:648] HybridClock initialized: now 1786280372095619 us; error 59 us; skew 500 ppm
I20260809 12:59:32.096032 4614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:32 dist-test-slave-gp6n krb5kdc[3443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280372, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:32.100286 4614 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 12:59:32.114653 4614 webserver.cc:460] Webserver started at http://127.21.235.65:37407/ using document root <none> and password file <none>
I20260809 12:59:32.115459 4614 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:32.115643 4614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 12:59:32 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280372, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
I20260809 12:59:32.147171 4614 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.004s sys 0.000s
I20260809 12:59:32.154981 4665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:32.156674 4614 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.005s
I20260809 12:59:32.156950 4614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "71340472ba4044feb19d3aa86f6e22b4"
format_stamp: "Formatted at 2026-08-09 12:59:28 on dist-test-slave-gp6n"
server_key: "89806b258376b09734c3f7ae5f0d33a0"
server_key_iv: "388d4f7c4547a620a96c9fad54bc711f"
server_key_version: "kuduclusterkey@0"
I20260809 12:59:32.158349 4614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:32.187942 4614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:32.195286 4614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:32.195665 4614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:32.205574 4614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:59:32.215981 4614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:59:32.216132 4614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:32.216253 4614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:59:32.216341 4614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:32.322944 4614 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:40763
I20260809 12:59:32.322999 4795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:40763 every 8 connection(s)
I20260809 12:59:32.324753 4614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:59:32.333289 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 4614
I20260809 12:59:32.334002 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.235.84:56787/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:40993
--local_ip_for_outbound_sockets=127.21.235.66
--tserver_master_addrs=127.21.235.126:40979
--webserver_port=39425
--webserver_interface=127.21.235.66
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.235.84:56787/kms with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:59:32 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280372, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:59:32.361500 4798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:32.328842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40979 (local address 127.21.235.65:33967)
0809 12:59:32.329912 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:32.329982 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:32.336778 (+ 6796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:32.344089 (+ 7311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:32.344103 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:32.344866 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:32.345482 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:32.345507 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:32.347217 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:32.347222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:32.348493 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:32.348505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:32.350530 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:32.350536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:32.350718 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:32.351384 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:32.351406 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:32.353456 (+ 2050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:32.356905 (+ 3449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:32.356914 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:32.356930 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:32.357412 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:32.357868 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:32.357873 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:32.357876 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:32.358084 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:32.358911 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:32.358918 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:32.359254 (+ 336us) client_negotiation.cc:769] Sending connection context
0809 12:59:32.359529 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 12:59:32.360089 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":1036,"threads_started":1}
I20260809 12:59:32.361496 4799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:32.329716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:33967 (local address 127.21.235.126:40979)
0809 12:59:32.331923 (+ 2207us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:32.331941 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:32.331993 (+ 52us) server_negotiation.cc:407] Connection header received
0809 12:59:32.343008 (+ 11015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:32.343023 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:32.343467 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:32.343896 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:32.345736 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:32.347052 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:32.348708 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:32.349938 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:32.350135 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:32.353699 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:32.353707 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:32.353720 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:32.353759 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:32.356752 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:32.357571 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:32.357578 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:32.357584 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:32.357743 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:32.358197 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:32.358205 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:32.358209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:32.358765 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:32.358974 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:32.359752 (+ 778us) server_negotiation.cc:299] Negotiation successful
0809 12:59:32.360151 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1950,"thread_start_us":1733,"threads_started":1}
I20260809 12:59:32.365406 4796 heartbeater.cc:347] Connected to a master server at 127.21.235.126:40979
I20260809 12:59:32.365741 4796 heartbeater.cc:464] Registering TS with master...
I20260809 12:59:32.366572 4796 heartbeater.cc:511] Master 127.21.235.126:40979 requested a full tablet report, sending...
W20260809 12:59:32.369445 4796 heartbeater.cc:644] Failed to heartbeat to 127.21.235.126:40979 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 12:59:32.369681 4796 heartbeater.cc:464] Registering TS with master...
I20260809 12:59:32.369944 4796 heartbeater.cc:511] Master 127.21.235.126:40979 requested a full tablet report, sending...
I20260809 12:59:32.371091 4796 heartbeater.cc:464] Registering TS with master...
I20260809 12:59:32.371361 4796 heartbeater.cc:511] Master 127.21.235.126:40979 requested a full tablet report, sending...
W20260809 12:59:32.372447 4796 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 12:59:32.751822 4800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:32.752144 4800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:32.752313 4800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:32.760265 4800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:32.760417 4800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:32.760613 4800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:59:32.771620 4800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.235.84:56787/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:40993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=39425
--tserver_master_addrs=127.21.235.126:40979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:32.773121 4800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:32.774493 4800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:32.787859 4815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:32.787882 4816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:32.788748 4818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:32.789237 4800 server_base.cc:1034] running on GCE node
I20260809 12:59:32.789984 4800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:32.791213 4800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:32.792469 4800 hybrid_clock.cc:648] HybridClock initialized: now 1786280372792399 us; error 49 us; skew 500 ppm
I20260809 12:59:32.792855 4800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:32 dist-test-slave-gp6n krb5kdc[3443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280372, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:32.797092 4800 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 12:59:32.805644 4800 webserver.cc:460] Webserver started at http://127.21.235.66:39425/ using document root <none> and password file <none>
I20260809 12:59:32.806632 4800 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:32.806834 4800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 12:59:32 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280372, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
I20260809 12:59:32.841066 4800 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 12:59:32.844288 4829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:32.845851 4800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:59:32.846051 4800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "c68f5f098a254a9092f92f51fe72e611"
format_stamp: "Formatted at 2026-08-09 12:59:29 on dist-test-slave-gp6n"
server_key: "0bec4f8cdd2c9db758e2e410f1244ed0"
server_key_iv: "9d3518c640a40e17248f17976447a799"
server_key_version: "kuduclusterkey@0"
I20260809 12:59:32.847237 4800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:32.863541 4800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:32.869622 4800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:32.870054 4800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:32.886881 4800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:59:32.903404 4800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:59:32.903636 4800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 12:59:32.903838 4800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:59:32.903986 4800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:32.967684 4800 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:40993
I20260809 12:59:32.967758 4956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:40993 every 8 connection(s)
I20260809 12:59:32.969751 4800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:59:32.971753 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 4800
I20260809 12:59:32.972553 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.235.84:56787/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:46441
--local_ip_for_outbound_sockets=127.21.235.67
--tserver_master_addrs=127.21.235.126:40979
--webserver_port=45981
--webserver_interface=127.21.235.67
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:39503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.235.84:56787/kms with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:59:33 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280372, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
Aug 09 12:59:33 dist-test-slave-gp6n krb5kdc[3443](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786280373, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:33.018774 4962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:32.981437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:44661 (local address 127.21.235.126:40979)
0809 12:59:32.995642 (+ 14205us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:32.995649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:32.995676 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:59:32.995746 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:32.995752 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:32.995853 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:32.995973 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:32.998481 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:32.999794 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.002876 (+ 3082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.003573 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.003680 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:33.008796 (+ 5116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:33.008804 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:33.008808 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:33.008851 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:33.012622 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:33.013602 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:33.013608 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:33.013611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:33.013776 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:33.014386 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:33.014391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:33.014394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:33.016806 (+ 2412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:33.016966 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:33.018109 (+ 1143us) server_negotiation.cc:299] Negotiation successful
0809 12:59:33.018361 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14051,"thread_start_us":18882,"threads_started":1}
I20260809 12:59:33.021786 4960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:32.980770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40979 (local address 127.21.235.66:44661)
0809 12:59:32.991387 (+ 10617us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:32.991452 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:32.993953 (+ 2501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:32.996534 (+ 2581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:32.996548 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:32.997247 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:32.998173 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:32.998212 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.000025 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.000030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:33.002690 (+ 2660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:33.002706 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.004098 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.004107 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:33.004430 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:33.005301 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:33.005334 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:33.008527 (+ 3193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:33.012796 (+ 4269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:33.012805 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:33.012823 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:33.013449 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:33.013937 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:33.013941 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:33.013944 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:33.014219 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:33.017064 (+ 2845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:33.017071 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:33.017629 (+ 558us) client_negotiation.cc:769] Sending connection context
0809 12:59:33.019377 (+ 1748us) client_negotiation.cc:240] Negotiation successful
0809 12:59:33.020028 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10244,"thread_start_us":13508,"threads_started":1}
I20260809 12:59:33.024961 4957 heartbeater.cc:347] Connected to a master server at 127.21.235.126:40979
I20260809 12:59:33.025324 4957 heartbeater.cc:464] Registering TS with master...
I20260809 12:59:33.026234 4957 heartbeater.cc:511] Master 127.21.235.126:40979 requested a full tablet report, sending...
I20260809 12:59:33.032550 4573 ts_manager.cc:194] Registered new tserver with Master: c68f5f098a254a9092f92f51fe72e611 (127.21.235.66:40993)
I20260809 12:59:33.034837 4573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:44661
I20260809 12:59:33.372632 4796 heartbeater.cc:464] Registering TS with master...
I20260809 12:59:33.373185 4796 heartbeater.cc:511] Master 127.21.235.126:40979 requested a full tablet report, sending...
I20260809 12:59:33.374567 4573 ts_manager.cc:194] Registered new tserver with Master: 71340472ba4044feb19d3aa86f6e22b4 (127.21.235.65:40763)
I20260809 12:59:33.375788 4573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:33967
Aug 09 12:59:33 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280373, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
W20260809 12:59:33.466939 4965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:33.467407 4965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:33.467574 4965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:33.479161 4965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:33.479343 4965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:33.479538 4965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:59:33.494984 4965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:39503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.235.84:56787/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:46441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=45981
--tserver_master_addrs=127.21.235.126:40979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:33.496793 4965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:33.498596 4965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:33.528080 4971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:33.528767 4972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:33.530084 4976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:33.530810 4965 server_base.cc:1034] running on GCE node
I20260809 12:59:33.531602 4965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:33.532847 4965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:33.534102 4965 hybrid_clock.cc:648] HybridClock initialized: now 1786280373534039 us; error 49 us; skew 500 ppm
I20260809 12:59:33.534544 4965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:33 dist-test-slave-gp6n krb5kdc[3443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280373, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:33.539017 4965 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 12:59:33.540999 4965 webserver.cc:460] Webserver started at http://127.21.235.67:45981/ using document root <none> and password file <none>
I20260809 12:59:33.542007 4965 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:33.542268 4965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 12:59:33 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280373, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
I20260809 12:59:33.571511 4965 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 12:59:33.575156 4985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:33.577040 4965 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260809 12:59:33.577221 4965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "83f898636ee04f7690161e948a6d0033"
format_stamp: "Formatted at 2026-08-09 12:59:30 on dist-test-slave-gp6n"
server_key: "7f9c207f51fc9476f391636ba19005f8"
server_key_iv: "700bd232c3c43b397ba5b2f7bb06694d"
server_key_version: "kuduclusterkey@0"
I20260809 12:59:33.578619 4965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:33.595849 4965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:33.600191 4965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:33.600481 4965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:33.601570 4965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:59:33.603190 4965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:59:33.603329 4965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:33.603468 4965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:59:33.603565 4965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:33.665904 4965 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:46441
I20260809 12:59:33.666003 5113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:46441 every 8 connection(s)
I20260809 12:59:33.667682 4965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:59:33.668031 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 4965
Aug 09 12:59:33 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280373, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:59:33.714146 5117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:33.669490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40648 (local address 127.21.235.126:40979)
0809 12:59:33.684741 (+ 15251us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:33.684747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:33.684770 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:59:33.684836 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:33.684841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:33.684927 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:33.685033 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:33.687579 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.688635 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.693932 (+ 5297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.694629 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.694760 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:33.704678 (+ 9918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:33.704686 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:33.704691 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:33.704732 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:33.708220 (+ 3488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:33.709148 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:33.709154 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:33.709158 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:33.709326 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:33.711623 (+ 2297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:33.711628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:33.711632 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:33.712030 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:33.712288 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:33.713519 (+ 1231us) server_negotiation.cc:299] Negotiation successful
0809 12:59:33.713748 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15126,"thread_start_us":14960,"threads_started":1}
I20260809 12:59:33.724423 5118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:33.671287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:41739 (local address 127.21.235.126:40979)
0809 12:59:33.685330 (+ 14043us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:33.685336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:33.685363 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:59:33.689549 (+ 4186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:33.689555 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:33.689628 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:33.689724 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:33.695859 (+ 6135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.697030 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.702494 (+ 5464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.703171 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.703274 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:33.716479 (+ 13205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:33.716486 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:33.716491 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:33.716527 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:33.720378 (+ 3851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:33.721291 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:33.721297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:33.721301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:33.721447 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:33.722000 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:33.722005 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:33.722008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:33.722381 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:33.722569 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:33.723851 (+ 1282us) server_negotiation.cc:299] Negotiation successful
0809 12:59:33.724073 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13933,"thread_start_us":12943,"threads_started":1}
I20260809 12:59:33.725436 5119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:33.671959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40979 (local address 127.21.235.67:41739)
0809 12:59:33.680202 (+ 8243us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:33.680252 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:33.689103 (+ 8851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:33.690753 (+ 1650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:33.690768 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:33.691412 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:33.692306 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:33.692333 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.697230 (+ 4897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.697235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:33.702280 (+ 5045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:33.702296 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.703695 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.703703 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:33.704023 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:33.709786 (+ 5763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:33.709817 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:33.716151 (+ 6334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:33.720534 (+ 4383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:33.720543 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:33.720562 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:33.721133 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:33.721568 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:33.721573 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:33.721576 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:33.721848 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:33.722648 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:33.722657 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:33.723072 (+ 415us) client_negotiation.cc:769] Sending connection context
0809 12:59:33.723399 (+ 327us) client_negotiation.cc:240] Negotiation successful
0809 12:59:33.723985 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7937,"thread_start_us":10345,"threads_started":1}
I20260809 12:59:33.727114 5114 heartbeater.cc:347] Connected to a master server at 127.21.235.126:40979
I20260809 12:59:33.727449 5114 heartbeater.cc:464] Registering TS with master...
I20260809 12:59:33.728356 5114 heartbeater.cc:511] Master 127.21.235.126:40979 requested a full tablet report, sending...
I20260809 12:59:33.730221 4572 ts_manager.cc:194] Registered new tserver with Master: 83f898636ee04f7690161e948a6d0033 (127.21.235.67:46441)
I20260809 12:59:33.731487 4572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:41739
I20260809 12:59:33.735087 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:59:33.763052 5118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:33.744335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40652 (local address 127.21.235.126:40979)
0809 12:59:33.744542 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:33.744549 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:33.746441 (+ 1892us) server_negotiation.cc:407] Connection header received
0809 12:59:33.746634 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:33.746640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:33.746754 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:33.746869 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:33.748882 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.750009 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.751988 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.752579 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.752670 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:33.756241 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:33.756247 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:33.756251 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:33.756290 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:33.759830 (+ 3540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:33.760666 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:33.760671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:33.760674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:33.760840 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:33.761329 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:33.761334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:33.761338 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:33.761740 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:33.761987 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:33.762464 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 12:59:33.762711 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 12:59:33.769872 4572 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40652:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:59:33.781668 4572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 09 12:59:33 dist-test-slave-gp6n krb5kdc[3443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280373, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for HTTP/127.21.235.84@KRBTEST.COM
I20260809 12:59:33.848480 5131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:33.814898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:46441 (local address 127.0.0.1:48832)
0809 12:59:33.815571 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:33.815621 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:33.815800 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:33.827742 (+ 11942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:33.827749 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:33.827783 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:59:33.828229 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:33.828244 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.836073 (+ 7829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.836079 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:33.838522 (+ 2443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:33.838540 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.844174 (+ 5634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.844181 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:33.844400 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:33.845188 (+ 788us) client_negotiation.cc:769] Sending connection context
0809 12:59:33.845309 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 12:59:33.845521 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":901,"threads_started":1}
I20260809 12:59:33.850023 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:33.815640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48832 (local address 127.21.235.67:46441)
0809 12:59:33.826113 (+ 10473us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:33.826121 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:33.826154 (+ 33us) server_negotiation.cc:407] Connection header received
0809 12:59:33.826251 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:33.826466 (+ 215us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:33.826748 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:33.828463 (+ 1715us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:59:33.828744 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.833053 (+ 4309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.839483 (+ 6430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.840901 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.849196 (+ 8295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:33.849483 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:33.849552 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 12:59:33.849716 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10294,"thread_start_us":10147,"threads_started":1}
I20260809 12:59:33.854617 5039 tablet_service.cc:1467] Processing CreateTablet for tablet 4ee9648dd5b24b6489cd9c74fae4fa5a (DEFAULT_TABLE table=test-table [id=d161ebac089943928d3c378e9c490afd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:59:33.856232 5039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ee9648dd5b24b6489cd9c74fae4fa5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:59:33.856905 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:33.826007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:40763 (local address 127.0.0.1:40336)
0809 12:59:33.847196 (+ 21189us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:33.847211 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:33.847349 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:33.848240 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:33.848245 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:33.848267 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:59:33.848643 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:33.848654 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.851821 (+ 3167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.851826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:33.853920 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:33.853936 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.856113 (+ 2177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.856119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:33.856304 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:33.856337 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 12:59:33.856429 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 12:59:33.856562 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21068,"thread_start_us":20904,"threads_started":1}
I20260809 12:59:33.857692 5136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:33.826719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40336 (local address 127.21.235.65:40763)
0809 12:59:33.844209 (+ 17490us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:33.844218 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:33.847322 (+ 3104us) server_negotiation.cc:407] Connection header received
0809 12:59:33.847532 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:33.847540 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:33.847786 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:33.847998 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:59:33.848830 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.851654 (+ 2824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.854156 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.855560 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.855701 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:33.855963 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:33.857208 (+ 1245us) server_negotiation.cc:299] Negotiation successful
0809 12:59:33.857373 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17290,"thread_start_us":17663,"threads_started":1}
I20260809 12:59:33.861982 4718 tablet_service.cc:1467] Processing CreateTablet for tablet 4ee9648dd5b24b6489cd9c74fae4fa5a (DEFAULT_TABLE table=test-table [id=d161ebac089943928d3c378e9c490afd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:59:33.863591 4718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ee9648dd5b24b6489cd9c74fae4fa5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:59:33.879601 5133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:33.818155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:40993 (local address 127.0.0.1:51960)
0809 12:59:33.843449 (+ 25294us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:33.843469 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:33.844756 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:33.846013 (+ 1257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:33.846021 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:33.846059 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:59:33.846572 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:33.846589 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.849289 (+ 2700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.849294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:33.860561 (+ 11267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:33.860576 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.878605 (+ 18029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.878613 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:33.878817 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:33.878857 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 12:59:33.879017 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 12:59:33.879184 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25151,"thread_start_us":26080,"threads_started":1}
I20260809 12:59:33.880494 5134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:33.818643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51960 (local address 127.21.235.66:40993)
0809 12:59:33.819707 (+ 1064us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:33.819720 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:33.843702 (+ 23982us) server_negotiation.cc:407] Connection header received
0809 12:59:33.845045 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:33.845059 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:33.845360 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:33.845646 (+ 286us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:59:33.847536 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.849072 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.870322 (+ 21250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:33.871870 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:33.872035 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:33.872240 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:33.879978 (+ 7738us) server_negotiation.cc:299] Negotiation successful
0809 12:59:33.880178 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":852,"thread_start_us":250,"threads_started":1}
I20260809 12:59:33.884536 4890 tablet_service.cc:1467] Processing CreateTablet for tablet 4ee9648dd5b24b6489cd9c74fae4fa5a (DEFAULT_TABLE table=test-table [id=d161ebac089943928d3c378e9c490afd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:59:33.885984 4890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ee9648dd5b24b6489cd9c74fae4fa5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:59:33.907524 5138 tablet_bootstrap.cc:492] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 71340472ba4044feb19d3aa86f6e22b4: Bootstrap starting.
I20260809 12:59:33.913738 5138 tablet_bootstrap.cc:654] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 71340472ba4044feb19d3aa86f6e22b4: Neither blocks nor log segments found. Creating new log.
I20260809 12:59:33.915434 5138 log.cc:824] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 71340472ba4044feb19d3aa86f6e22b4: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:33.916992 5140 tablet_bootstrap.cc:492] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033: Bootstrap starting.
I20260809 12:59:33.922072 5140 tablet_bootstrap.cc:654] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033: Neither blocks nor log segments found. Creating new log.
I20260809 12:59:33.923389 5140 log.cc:824] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:33.925737 5138 tablet_bootstrap.cc:492] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 71340472ba4044feb19d3aa86f6e22b4: No bootstrap required, opened a new log
I20260809 12:59:33.926087 5138 ts_tablet_manager.cc:1397] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 71340472ba4044feb19d3aa86f6e22b4: Time spent bootstrapping tablet: real 0.019s user 0.014s sys 0.000s
I20260809 12:59:33.929998 5139 tablet_bootstrap.cc:492] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611: Bootstrap starting.
I20260809 12:59:33.933301 5140 tablet_bootstrap.cc:492] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033: No bootstrap required, opened a new log
I20260809 12:59:33.933612 5140 ts_tablet_manager.cc:1397] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.006s
I20260809 12:59:33.934944 5139 tablet_bootstrap.cc:654] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611: Neither blocks nor log segments found. Creating new log.
I20260809 12:59:33.938602 5138 raft_consensus.cc:348] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 71340472ba4044feb19d3aa86f6e22b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f898636ee04f7690161e948a6d0033" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 46441 } } peers { permanent_uuid: "c68f5f098a254a9092f92f51fe72e611" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40993 } } peers { permanent_uuid: "71340472ba4044feb19d3aa86f6e22b4" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40763 } }
I20260809 12:59:33.939059 5138 raft_consensus.cc:374] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 71340472ba4044feb19d3aa86f6e22b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:59:33.939162 5138 raft_consensus.cc:729] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 71340472ba4044feb19d3aa86f6e22b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71340472ba4044feb19d3aa86f6e22b4, State: Initialized, Role: FOLLOWER
I20260809 12:59:33.939925 5138 consensus_queue.cc:260] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 71340472ba4044feb19d3aa86f6e22b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f898636ee04f7690161e948a6d0033" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 46441 } } peers { permanent_uuid: "c68f5f098a254a9092f92f51fe72e611" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40993 } } peers { permanent_uuid: "71340472ba4044feb19d3aa86f6e22b4" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40763 } }
I20260809 12:59:33.944276 4796 heartbeater.cc:503] Master 127.21.235.126:40979 was elected leader, sending a full tablet report...
I20260809 12:59:33.944631 5140 raft_consensus.cc:348] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f898636ee04f7690161e948a6d0033" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 46441 } } peers { permanent_uuid: "c68f5f098a254a9092f92f51fe72e611" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40993 } } peers { permanent_uuid: "71340472ba4044feb19d3aa86f6e22b4" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40763 } }
I20260809 12:59:33.945050 5138 ts_tablet_manager.cc:1428] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 71340472ba4044feb19d3aa86f6e22b4: Time spent starting tablet: real 0.019s user 0.015s sys 0.004s
I20260809 12:59:33.945075 5140 raft_consensus.cc:374] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:59:33.945070 5139 log.cc:824] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:33.945182 5140 raft_consensus.cc:729] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f898636ee04f7690161e948a6d0033, State: Initialized, Role: FOLLOWER
I20260809 12:59:33.945796 5140 consensus_queue.cc:260] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f898636ee04f7690161e948a6d0033" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 46441 } } peers { permanent_uuid: "c68f5f098a254a9092f92f51fe72e611" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40993 } } peers { permanent_uuid: "71340472ba4044feb19d3aa86f6e22b4" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40763 } }
I20260809 12:59:33.951623 5139 tablet_bootstrap.cc:492] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611: No bootstrap required, opened a new log
I20260809 12:59:33.951902 5139 ts_tablet_manager.cc:1397] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.005s
I20260809 12:59:33.951932 5140 ts_tablet_manager.cc:1428] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033: Time spent starting tablet: real 0.018s user 0.011s sys 0.006s
I20260809 12:59:33.952253 5114 heartbeater.cc:503] Master 127.21.235.126:40979 was elected leader, sending a full tablet report...
I20260809 12:59:33.962236 5139 raft_consensus.cc:348] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f898636ee04f7690161e948a6d0033" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 46441 } } peers { permanent_uuid: "c68f5f098a254a9092f92f51fe72e611" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40993 } } peers { permanent_uuid: "71340472ba4044feb19d3aa86f6e22b4" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40763 } }
I20260809 12:59:33.962796 5139 raft_consensus.cc:374] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:59:33.962930 5139 raft_consensus.cc:729] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c68f5f098a254a9092f92f51fe72e611, State: Initialized, Role: FOLLOWER
I20260809 12:59:33.963847 5139 consensus_queue.cc:260] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f898636ee04f7690161e948a6d0033" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 46441 } } peers { permanent_uuid: "c68f5f098a254a9092f92f51fe72e611" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40993 } } peers { permanent_uuid: "71340472ba4044feb19d3aa86f6e22b4" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40763 } }
W20260809 12:59:33.965282 5120 tablet.cc:2369] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:59:33.967206 4957 heartbeater.cc:503] Master 127.21.235.126:40979 was elected leader, sending a full tablet report...
I20260809 12:59:33.972085 5139 ts_tablet_manager.cc:1428] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611: Time spent starting tablet: real 0.020s user 0.004s sys 0.009s
W20260809 12:59:33.979532 4958 tablet.cc:2369] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:59:34.022198 5145 raft_consensus.cc:482] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:59:34.022614 5145 raft_consensus.cc:504] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f898636ee04f7690161e948a6d0033" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 46441 } } peers { permanent_uuid: "c68f5f098a254a9092f92f51fe72e611" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40993 } } peers { permanent_uuid: "71340472ba4044feb19d3aa86f6e22b4" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40763 } }
I20260809 12:59:34.024873 5145 leader_election.cc:288] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c68f5f098a254a9092f92f51fe72e611 (127.21.235.66:40993), 71340472ba4044feb19d3aa86f6e22b4 (127.21.235.65:40763)
I20260809 12:59:34.033739 5119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:34.025472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:40993 (local address 127.21.235.67:53579)
0809 12:59:34.025872 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:34.025893 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:34.026014 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:34.027107 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:34.027112 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:34.027138 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:59:34.027352 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:34.027365 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:34.028874 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:34.028880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:34.030979 (+ 2099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:34.030995 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:34.032974 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:34.032981 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:34.033183 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:34.033218 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 12:59:34.033323 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 12:59:34.033473 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276}
I20260809 12:59:34.034477 5134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:34.026379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:53579 (local address 127.21.235.66:40993)
0809 12:59:34.026611 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:34.026617 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:34.026638 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:59:34.026725 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:34.026731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:34.026805 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:34.026918 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:59:34.027541 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:34.028688 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:34.031178 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:34.032487 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:34.032657 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:34.032737 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:34.034036 (+ 1299us) server_negotiation.cc:299] Negotiation successful
0809 12:59:34.034176 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260809 12:59:34.035641 4912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ee9648dd5b24b6489cd9c74fae4fa5a" candidate_uuid: "83f898636ee04f7690161e948a6d0033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c68f5f098a254a9092f92f51fe72e611" is_pre_election: true
I20260809 12:59:34.036237 4912 raft_consensus.cc:2454] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f898636ee04f7690161e948a6d0033 in term 0.
I20260809 12:59:34.037051 4992 leader_election.cc:302] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f898636ee04f7690161e948a6d0033, c68f5f098a254a9092f92f51fe72e611; no voters:
I20260809 12:59:34.037528 5145 raft_consensus.cc:2781] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:59:34.037678 5145 raft_consensus.cc:482] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:59:34.037786 5145 raft_consensus.cc:3037] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:59:34.041605 5145 raft_consensus.cc:504] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f898636ee04f7690161e948a6d0033" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 46441 } } peers { permanent_uuid: "c68f5f098a254a9092f92f51fe72e611" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40993 } } peers { permanent_uuid: "71340472ba4044feb19d3aa86f6e22b4" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40763 } }
I20260809 12:59:34.042918 4912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ee9648dd5b24b6489cd9c74fae4fa5a" candidate_uuid: "83f898636ee04f7690161e948a6d0033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c68f5f098a254a9092f92f51fe72e611"
I20260809 12:59:34.043042 5136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:34.026091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:56605 (local address 127.21.235.65:40763)
0809 12:59:34.026322 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:34.026328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:34.033046 (+ 6718us) server_negotiation.cc:407] Connection header received
0809 12:59:34.035079 (+ 2033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:34.035086 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:34.035204 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:34.035277 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:59:34.036215 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:34.037350 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:34.040483 (+ 3133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:34.041704 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:34.041840 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:34.041912 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:34.042652 (+ 740us) server_negotiation.cc:299] Negotiation successful
0809 12:59:34.042789 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 12:59:34.043138 4912 raft_consensus.cc:3037] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:59:34.043963 5147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:34.025557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:40763 (local address 127.21.235.67:56605)
0809 12:59:34.032918 (+ 7361us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:34.032943 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:34.034747 (+ 1804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:34.035601 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:34.035606 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:34.035634 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:59:34.036015 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:34.036033 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:34.037919 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:34.037925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:34.040257 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:34.040289 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:34.042214 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:34.042220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:34.042455 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:34.042489 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 12:59:34.043482 (+ 993us) client_negotiation.cc:240] Negotiation successful
0809 12:59:34.043630 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7223,"thread_start_us":7042,"threads_started":1}
I20260809 12:59:34.045154 4743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ee9648dd5b24b6489cd9c74fae4fa5a" candidate_uuid: "83f898636ee04f7690161e948a6d0033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71340472ba4044feb19d3aa86f6e22b4" is_pre_election: true
I20260809 12:59:34.045773 4743 raft_consensus.cc:2454] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 71340472ba4044feb19d3aa86f6e22b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f898636ee04f7690161e948a6d0033 in term 0.
I20260809 12:59:34.046290 4912 raft_consensus.cc:2454] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f898636ee04f7690161e948a6d0033 in term 1.
I20260809 12:59:34.046741 4992 leader_election.cc:302] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f898636ee04f7690161e948a6d0033, c68f5f098a254a9092f92f51fe72e611; no voters:
I20260809 12:59:34.047492 5149 raft_consensus.cc:2781] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:59:34.047952 4743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ee9648dd5b24b6489cd9c74fae4fa5a" candidate_uuid: "83f898636ee04f7690161e948a6d0033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71340472ba4044feb19d3aa86f6e22b4"
I20260809 12:59:34.048146 4743 raft_consensus.cc:3037] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 71340472ba4044feb19d3aa86f6e22b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:59:34.051215 4743 raft_consensus.cc:2454] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 71340472ba4044feb19d3aa86f6e22b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f898636ee04f7690161e948a6d0033 in term 1.
I20260809 12:59:34.051757 5145 leader_election.cc:288] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [CANDIDATE]: Term 1 election: Requested vote from peers c68f5f098a254a9092f92f51fe72e611 (127.21.235.66:40993), 71340472ba4044feb19d3aa86f6e22b4 (127.21.235.65:40763)
I20260809 12:59:34.052847 5149 raft_consensus.cc:686] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [term 1 LEADER]: Becoming Leader. State: Replica: 83f898636ee04f7690161e948a6d0033, State: Running, Role: LEADER
I20260809 12:59:34.053509 5149 consensus_queue.cc:237] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f898636ee04f7690161e948a6d0033" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 46441 } } peers { permanent_uuid: "c68f5f098a254a9092f92f51fe72e611" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40993 } } peers { permanent_uuid: "71340472ba4044feb19d3aa86f6e22b4" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40763 } }
I20260809 12:59:34.064256 4573 catalog_manager.cc:5495] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f898636ee04f7690161e948a6d0033 (127.21.235.67). New cstate: current_term: 1 leader_uuid: "83f898636ee04f7690161e948a6d0033" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f898636ee04f7690161e948a6d0033" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 46441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c68f5f098a254a9092f92f51fe72e611" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71340472ba4044feb19d3aa86f6e22b4" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40763 } health_report { overall_health: UNKNOWN } } }
W20260809 12:59:34.084864 4797 tablet.cc:2369] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 71340472ba4044feb19d3aa86f6e22b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:59:34.128489 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:34.122541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48846 (local address 127.21.235.67:46441)
0809 12:59:34.122745 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:34.122752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:34.122773 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:59:34.122919 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:34.122924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:34.123024 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:34.123185 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:59:34.123513 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:34.124579 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:34.125979 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:34.126615 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:34.126748 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:34.127223 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:59:34.127931 (+ 708us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:59:34.128077 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:34.128131 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 12:59:34.128213 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 12:59:34.137529 4743 raft_consensus.cc:1261] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 71340472ba4044feb19d3aa86f6e22b4 [term 1 FOLLOWER]: Refusing update from remote peer 83f898636ee04f7690161e948a6d0033: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:59:34.137571 4912 raft_consensus.cc:1261] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611 [term 1 FOLLOWER]: Refusing update from remote peer 83f898636ee04f7690161e948a6d0033: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:59:34.138602 5150 consensus_queue.cc:1035] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71340472ba4044feb19d3aa86f6e22b4" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 40763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:59:34.138975 5149 consensus_queue.cc:1035] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c68f5f098a254a9092f92f51fe72e611" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:59:34.170353 5164 mvcc.cc:205] Tried to move back new op lower bound from 7316604412456951808 to 7316604412128739328. Current Snapshot: MvccSnapshot[applied={T|T < 7316604412456951808}]
I20260809 12:59:34.180109 4573 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:59:34.180426 4573 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:59:34.184809 4573 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6738c5c21293491cbd85d67c7f04ecb4: Sending DeleteTablet for 3 replicas of tablet 4ee9648dd5b24b6489cd9c74fae4fa5a
I20260809 12:59:34.186065 4890 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ee9648dd5b24b6489cd9c74fae4fa5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:59:34 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:51960
I20260809 12:59:34.186098 4718 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ee9648dd5b24b6489cd9c74fae4fa5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:59:34 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:40336
I20260809 12:59:34.186852 5039 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ee9648dd5b24b6489cd9c74fae4fa5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:59:34 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:48832
I20260809 12:59:34.199241 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 4614
I20260809 12:59:34.240926 5136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:34.201481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40342 (local address 127.21.235.65:40763)
0809 12:59:34.201663 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:34.201669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:34.201687 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:59:34.201749 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:34.201755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:34.201832 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:34.201950 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:34.210058 (+ 8108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:34.212315 (+ 2257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:34.215167 (+ 2852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:34.216167 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:34.216295 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:34.226968 (+ 10673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:34.226991 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:34.227012 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:34.227061 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:34.232372 (+ 5311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:34.233825 (+ 1453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:34.233835 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:34.233839 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:34.234016 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:34.235918 (+ 1902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:34.235925 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:34.235932 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:34.236605 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:34.237052 (+ 447us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:34.240250 (+ 3198us) server_negotiation.cc:299] Negotiation successful
0809 12:59:34.240506 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 12:59:34.241675 4790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:34.248029 5173 tablet_replica.cc:330] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033: stopping tablet replica
I20260809 12:59:34.248618 5173 raft_consensus.cc:2229] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:59:34.249389 5173 raft_consensus.cc:2258] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:59:34.251868 5173 ts_tablet_manager.cc:1905] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:59:34.253907 5172 tablet_replica.cc:330] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611: stopping tablet replica
I20260809 12:59:34.254366 5172 raft_consensus.cc:2229] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:59:34.254980 5172 raft_consensus.cc:2258] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:59:34.257368 5172 ts_tablet_manager.cc:1905] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:59:34.261171 5173 ts_tablet_manager.cc:1918] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:59:34.261364 5173 log.cc:1196] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/4ee9648dd5b24b6489cd9c74fae4fa5a
I20260809 12:59:34.261951 5173 ts_tablet_manager.cc:1939] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 83f898636ee04f7690161e948a6d0033: Deleting consensus metadata
I20260809 12:59:34.265571 5172 ts_tablet_manager.cc:1918] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:59:34.265739 5172 log.cc:1196] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/4ee9648dd5b24b6489cd9c74fae4fa5a
I20260809 12:59:34.266278 5172 ts_tablet_manager.cc:1939] T 4ee9648dd5b24b6489cd9c74fae4fa5a P c68f5f098a254a9092f92f51fe72e611: Deleting consensus metadata
I20260809 12:59:34.268219 4559 catalog_manager.cc:4841] TS c68f5f098a254a9092f92f51fe72e611 (127.21.235.66:40993): tablet 4ee9648dd5b24b6489cd9c74fae4fa5a (table test-table [id=d161ebac089943928d3c378e9c490afd]) successfully deleted
I20260809 12:59:34.271667 4559 catalog_manager.cc:4841] TS 83f898636ee04f7690161e948a6d0033 (127.21.235.67:46441): tablet 4ee9648dd5b24b6489cd9c74fae4fa5a (table test-table [id=d161ebac089943928d3c378e9c490afd]) successfully deleted
I20260809 12:59:34.422969 5171 tablet_replica.cc:330] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 71340472ba4044feb19d3aa86f6e22b4: stopping tablet replica
I20260809 12:59:34.423465 5171 raft_consensus.cc:2229] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 71340472ba4044feb19d3aa86f6e22b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:59:34.423758 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 4614
I20260809 12:59:34.424015 5171 raft_consensus.cc:2258] T 4ee9648dd5b24b6489cd9c74fae4fa5a P 71340472ba4044feb19d3aa86f6e22b4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 12:59:34.456113 4555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.235.65:40763 (error 108)
I20260809 12:59:34.457579 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 4800
I20260809 12:59:34.473599 5176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:34.458221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:40763 (local address 127.0.0.1:40352)
0809 12:59:34.473211 (+ 14990us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:34.473368 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:40763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14813,"threads_started":1}
W20260809 12:59:34.474025 4555 catalog_manager.cc:4584] TS 71340472ba4044feb19d3aa86f6e22b4 (127.21.235.65:40763): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4ee9648dd5b24b6489cd9c74fae4fa5a: Network error: Client connection negotiation failed: client connection to 127.21.235.65:40763: connect: Connection refused (error 111)
I20260809 12:59:34.475646 5134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:34.458829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51972 (local address 127.21.235.66:40993)
0809 12:59:34.458988 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:34.458994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:34.459019 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:59:34.459082 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:34.459086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:34.459147 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:34.459273 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:34.460966 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:34.462017 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:34.463865 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:34.464413 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:34.464522 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:34.468182 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:34.468199 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:34.468216 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:34.468263 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:34.471921 (+ 3658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:34.472761 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:34.472770 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:34.472774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:34.472940 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:34.473349 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:34.473353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:34.473356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:34.474135 (+ 779us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:34.474555 (+ 420us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:34.475066 (+ 511us) server_negotiation.cc:299] Negotiation successful
0809 12:59:34.475282 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 12:59:34.476372 4952 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:34.527945 5176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:34.527341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:40763 (local address 127.0.0.1:40362)
0809 12:59:34.527647 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:34.527773 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:40763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
I20260809 12:59:34.529228 5176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:34.528737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:40763 (local address 127.0.0.1:40374)
0809 12:59:34.528986 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:34.529073 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:40763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260809 12:59:34.589874 5176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:34.589255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:40763 (local address 127.0.0.1:40386)
0809 12:59:34.589567 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:34.589673 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:40763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260809 12:59:34.591261 5176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:34.590827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:40763 (local address 127.0.0.1:40388)
0809 12:59:34.591028 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:34.591111 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:40763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260809 12:59:34.625653 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 4800
I20260809 12:59:34.654353 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 4965
I20260809 12:59:34.672412 5176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:34.671845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:40763 (local address 127.0.0.1:40398)
0809 12:59:34.672133 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:34.672229 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:40763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
I20260809 12:59:34.673578 5176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:34.673101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:40763 (local address 127.0.0.1:40400)
0809 12:59:34.673346 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:34.673425 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:40763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260809 12:59:34.681389 5178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:34.655327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48858 (local address 127.21.235.67:46441)
0809 12:59:34.659100 (+ 3773us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:34.659109 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:34.659137 (+ 28us) server_negotiation.cc:407] Connection header received
0809 12:59:34.659209 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:34.659217 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:34.659310 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:34.659460 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:34.661323 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:34.662745 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:34.664650 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:34.665468 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:34.666390 (+ 922us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:34.670454 (+ 4064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:34.670471 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:34.670490 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:34.670553 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:34.677024 (+ 6471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:34.677988 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:34.677997 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:34.678001 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:34.678237 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:34.678756 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:34.678761 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:34.678764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:34.679229 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:34.680532 (+ 1303us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:34.680792 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 12:59:34.681031 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3615,"thread_start_us":3448,"threads_started":1}
I20260809 12:59:34.682113 5108 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:34.808145 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 4965
I20260809 12:59:34.830525 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 4535
I20260809 12:59:34.832038 4606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:34.949447 5176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:34.948321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:40763 (local address 127.0.0.1:40406)
0809 12:59:34.949135 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:34.949236 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:40763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463}
I20260809 12:59:34.949783 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 4535
I20260809 12:59:34.977231 22445 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 12:59:35.322743 22445 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 12:59:35.322999 22445 mini_ranger.cc:66] Stopping Ranger...
I20260809 12:59:35.668242 22445 mini_ranger.cc:68] Stopped Ranger
2026-08-09 12:59:35.668 UTC [3479] LOG: received smart shutdown request
2026-08-09 12:59:35.671 UTC [3479] LOG: background worker "logical replication launcher" (PID 3489) exited with exit code 1
2026-08-09 12:59:35.672 UTC [3484] LOG: shutting down
2026-08-09 12:59:35.846 UTC [3479] LOG: database system is shut down
2026-08-09T12:59:35Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54877 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:59:36 dist-test-slave-gp6n krb5kdc[5190](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:59:36 dist-test-slave-gp6n krb5kdc[5190](info): set up 2 sockets
Aug 09 12:59:36 dist-test-slave-gp6n krb5kdc[5190](info): commencing operation
krb5kdc: starting...
W20260809 12:59:37.152419 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:59:37 dist-test-slave-gp6n krb5kdc[5190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:59:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:59:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 12:59:37.413901 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:35917
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:40011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:35917,127.21.235.125:44077,127.21.235.124:36983
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:59:37.756764 5214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:37.757121 5214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:37.757282 5214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:37.764576 5214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:59:37.764726 5214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:37.764828 5214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:37.764935 5214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:59:37.765012 5214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:59:37.774879 5214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:40011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:35917,127.21.235.125:44077,127.21.235.124:36983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:35917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:37.776176 5214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:37.777498 5214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:37.789556 5219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:37.789655 5222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:37.789595 5220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:37.790347 5214 server_base.cc:1034] running on GCE node
I20260809 12:59:37.791090 5214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:37.792385 5214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:37.793594 5214 hybrid_clock.cc:648] HybridClock initialized: now 1786280377793559 us; error 29 us; skew 500 ppm
I20260809 12:59:37.793938 5214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:37 dist-test-slave-gp6n krb5kdc[5190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280377, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:37.800880 5214 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:59:37.802361 5214 webserver.cc:460] Webserver started at http://127.21.235.126:41711/ using document root <none> and password file <none>
I20260809 12:59:37.803094 5214 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:37.803238 5214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:37.803546 5214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:59:37.806560 5214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "e61d7fbe0129421ea5e64ecbea21c43f"
format_stamp: "Formatted at 2026-08-09 12:59:37 on dist-test-slave-gp6n"
server_key: "2245edff8b6e826e9504fe2b834a7d6b"
server_key_iv: "5349890f34b7cdae621faa5d26356fde"
server_key_version: "encryptionkey@0"
I20260809 12:59:37.807367 5214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "e61d7fbe0129421ea5e64ecbea21c43f"
format_stamp: "Formatted at 2026-08-09 12:59:37 on dist-test-slave-gp6n"
server_key: "2245edff8b6e826e9504fe2b834a7d6b"
server_key_iv: "5349890f34b7cdae621faa5d26356fde"
server_key_version: "encryptionkey@0"
I20260809 12:59:37.812331 5214 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 12:59:37.815918 5229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:37.817224 5214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:59:37.817401 5214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "e61d7fbe0129421ea5e64ecbea21c43f"
format_stamp: "Formatted at 2026-08-09 12:59:37 on dist-test-slave-gp6n"
server_key: "2245edff8b6e826e9504fe2b834a7d6b"
server_key_iv: "5349890f34b7cdae621faa5d26356fde"
server_key_version: "encryptionkey@0"
I20260809 12:59:37.817595 5214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:37.832839 5214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:37.837417 5214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:37.837757 5214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:37.856252 5214 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:35917
I20260809 12:59:37.856290 5283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:35917 every 8 connection(s)
I20260809 12:59:37.857645 5214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:59:37.859953 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 5214
I20260809 12:59:37.860205 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:59:37.860859 22445 external_mini_cluster.cc:1428] Setting key 086fc7d5a144a844bf2ed401a9605741
I20260809 12:59:37.861521 5284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:59:37.867964 5284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:59:37 dist-test-slave-gp6n krb5kdc[5190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280377, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.235.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.125@KRBTEST.COM" created.
I20260809 12:59:37.890149 5288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:37.871120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38648 (local address 127.21.235.126:35917)
0809 12:59:37.872785 (+ 1665us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:37.872793 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:37.872841 (+ 48us) server_negotiation.cc:407] Connection header received
0809 12:59:37.874385 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:37.874391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:37.874687 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:37.874900 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:37.876162 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:37.877340 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:37.878827 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:37.879370 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:37.879590 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:37.882931 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:37.882939 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:37.882951 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:37.882980 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:37.885809 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:37.886577 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:37.886585 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:37.886587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:37.886785 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:37.887201 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:37.887214 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:37.887218 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:37.887848 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:37.888015 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:37.888770 (+ 755us) server_negotiation.cc:299] Negotiation successful
0809 12:59:37.889017 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1449,"spinlock_wait_cycles":16128,"thread_start_us":1235,"threads_started":1}
I20260809 12:59:37.890252 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:37.871118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:35917 (local address 127.0.0.1:38648)
0809 12:59:37.871970 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:37.872024 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:37.874016 (+ 1992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:37.874910 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:37.874916 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:37.875622 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:37.875963 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:37.875981 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:37.877506 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:37.877513 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:37.878687 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:37.878700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:37.879604 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:37.879611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:37.879790 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:37.880488 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:37.880509 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:37.882712 (+ 2203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:37.885947 (+ 3235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:37.885958 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:37.885969 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:37.886350 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:37.886892 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:37.886895 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:37.886897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:37.887067 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:37.888002 (+ 935us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:37.888013 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:37.888290 (+ 277us) client_negotiation.cc:769] Sending connection context
0809 12:59:37.888588 (+ 298us) client_negotiation.cc:240] Negotiation successful
0809 12:59:37.889247 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":109,"thread_start_us":370,"threads_started":1}
I20260809 12:59:37.893229 5284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:59:37.894389 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:37.894013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44077 (local address 127.0.0.1:51456)
0809 12:59:37.894184 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:37.894285 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260809 12:59:37.894840 5232 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44077: connect: Connection refused (error 111)
I20260809 12:59:37.896382 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:37.896059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44077 (local address 127.0.0.1:51466)
0809 12:59:37.896201 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:37.896267 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 12:59:37.896662 5284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.125:44077: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44077: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.125.keytab.
Entry for principal kudu/127.21.235.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.125.keytab.
I20260809 12:59:37.925947 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.125:44077
--webserver_interface=127.21.235.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.125.keytab
--principal=kudu/127.21.235.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:40011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:35917,127.21.235.125:44077,127.21.235.124:36983
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:59:37.946038 5284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } attempt: 1
I20260809 12:59:37.947459 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:37.946945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44077 (local address 127.0.0.1:51482)
0809 12:59:37.947168 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:37.947314 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260809 12:59:37.948650 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:37.948301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44077 (local address 127.0.0.1:51494)
0809 12:59:37.948453 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:37.948525 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 12:59:37.948931 5284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.125:44077: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44077: connect: Connection refused (error 111)
I20260809 12:59:38.017189 5284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } attempt: 2
I20260809 12:59:38.018587 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.018028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44077 (local address 127.0.0.1:51502)
0809 12:59:38.018282 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.018381 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260809 12:59:38.019603 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.019233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44077 (local address 127.0.0.1:51512)
0809 12:59:38.019404 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.019471 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 12:59:38.019830 5284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.125:44077: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44077: connect: Connection refused (error 111)
I20260809 12:59:38.111102 5284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } attempt: 3
I20260809 12:59:38.112452 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.111947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44077 (local address 127.0.0.1:51528)
0809 12:59:38.112185 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.112311 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260809 12:59:38.113580 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.113161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44077 (local address 127.0.0.1:51536)
0809 12:59:38.113332 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.113416 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 12:59:38.113794 5284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.125:44077: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44077: connect: Connection refused (error 111)
I20260809 12:59:38.257081 5284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } attempt: 4
I20260809 12:59:38.258374 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.257783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44077 (local address 127.0.0.1:51550)
0809 12:59:38.258120 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.258214 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260809 12:59:38.259553 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.259163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44077 (local address 127.0.0.1:51560)
0809 12:59:38.259322 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.259426 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 12:59:38.259760 5284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.125:44077: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44077: connect: Connection refused (error 111)
W20260809 12:59:38.276154 5291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:38.276463 5291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:38.276656 5291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:38.283854 5291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:59:38.283948 5291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:38.284060 5291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:38.284145 5291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:59:38.284211 5291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:59:38.293955 5291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:40011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:35917,127.21.235.125:44077,127.21.235.124:36983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.125.keytab
--principal=kudu/127.21.235.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.125:44077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.235.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:38.295378 5291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:38.296696 5291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:38.308310 5299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:38.308432 5298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:38.308406 5301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:38.309496 5291 server_base.cc:1034] running on GCE node
I20260809 12:59:38.310179 5291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:38.311596 5291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:38.312793 5291 hybrid_clock.cc:648] HybridClock initialized: now 1786280378312756 us; error 31 us; skew 500 ppm
I20260809 12:59:38.313126 5291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:38 dist-test-slave-gp6n krb5kdc[5190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280378, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:38.316413 5291 init.cc:377] Logged in from keytab as kudu/127.21.235.125@KRBTEST.COM (short username kudu)
I20260809 12:59:38.317764 5291 webserver.cc:460] Webserver started at http://127.21.235.125:45767/ using document root <none> and password file <none>
I20260809 12:59:38.318521 5291 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:38.318665 5291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:38.318953 5291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:59:38.321998 5291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-1/data/instance:
uuid: "2dbb7244f0bb4224a3981b6c2243ed13"
format_stamp: "Formatted at 2026-08-09 12:59:38 on dist-test-slave-gp6n"
server_key: "e4ac3e63f1829bcc727a096bb8f79700"
server_key_iv: "617a847954cffc8de1d994ae1771ad3c"
server_key_version: "encryptionkey@0"
I20260809 12:59:38.322902 5291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-1/wal/instance:
uuid: "2dbb7244f0bb4224a3981b6c2243ed13"
format_stamp: "Formatted at 2026-08-09 12:59:38 on dist-test-slave-gp6n"
server_key: "e4ac3e63f1829bcc727a096bb8f79700"
server_key_iv: "617a847954cffc8de1d994ae1771ad3c"
server_key_version: "encryptionkey@0"
I20260809 12:59:38.327895 5291 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:59:38.331267 5308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:38.332657 5291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 12:59:38.332849 5291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-1/wal
uuid: "2dbb7244f0bb4224a3981b6c2243ed13"
format_stamp: "Formatted at 2026-08-09 12:59:38 on dist-test-slave-gp6n"
server_key: "e4ac3e63f1829bcc727a096bb8f79700"
server_key_iv: "617a847954cffc8de1d994ae1771ad3c"
server_key_version: "encryptionkey@0"
I20260809 12:59:38.333052 5291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:38.361008 5291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:38.365556 5291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:38.365877 5291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:38.384552 5291 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.125:44077
I20260809 12:59:38.384569 5361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.125:44077 every 8 connection(s)
I20260809 12:59:38.385895 5291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-1/data/info.pb
I20260809 12:59:38.389559 5362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:59:38.393072 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 5291
I20260809 12:59:38.393275 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-1/wal/instance
I20260809 12:59:38.393824 22445 external_mini_cluster.cc:1428] Setting key ce861449dba8b1e65850234192ddbd2a
I20260809 12:59:38.397701 5362 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:59:38 dist-test-slave-gp6n krb5kdc[5190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280378, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.125@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.235.124@KRBTEST.COM; defaulting to no policy
I20260809 12:59:38.422433 5365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.401098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38658 (local address 127.21.235.126:35917)
0809 12:59:38.401648 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:38.401657 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:38.402786 (+ 1129us) server_negotiation.cc:407] Connection header received
0809 12:59:38.404913 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:38.404919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:38.405021 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:38.405156 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:38.407459 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.408222 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.411096 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.411530 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.411633 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:38.416317 (+ 4684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:38.416323 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:38.416326 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:38.416372 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:38.419257 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.420036 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.420042 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.420044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.420166 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.420605 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.420609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.420611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.420925 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:38.421105 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:38.421797 (+ 692us) server_negotiation.cc:299] Negotiation successful
0809 12:59:38.422017 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":253,"threads_started":1}
I20260809 12:59:38.423588 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.401736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:35917 (local address 127.0.0.1:38658)
0809 12:59:38.402602 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.402652 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:38.404602 (+ 1950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:38.405427 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:38.405436 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:38.406298 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:38.407226 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:38.407256 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.408421 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.408431 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:38.410929 (+ 2498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:38.410950 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.411799 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.411810 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:38.412222 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:38.413068 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:38.413107 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:38.416056 (+ 2949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:38.419417 (+ 3361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:38.419426 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:38.419442 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:38.419881 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:38.420281 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:38.420286 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:38.420289 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:38.420472 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:38.421068 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:38.421076 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:38.421421 (+ 345us) client_negotiation.cc:769] Sending connection context
0809 12:59:38.421687 (+ 266us) client_negotiation.cc:240] Negotiation successful
0809 12:59:38.422224 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":360,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.124@KRBTEST.COM" created.
I20260809 12:59:38.425557 5362 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:59:38 dist-test-slave-gp6n krb5kdc[5190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280378, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.125@KRBTEST.COM for kudu/127.21.235.125@KRBTEST.COM
I20260809 12:59:38.440915 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.426535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44077 (local address 127.0.0.1:51562)
0809 12:59:38.426723 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.426745 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:38.426870 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:38.427578 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:38.427584 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:38.428025 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:38.428156 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:38.428165 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.429495 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.429501 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:38.430330 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:38.430340 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.431642 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.431647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:38.431855 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:38.432439 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:38.432456 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:38.434673 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:38.438333 (+ 3660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:38.438341 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:38.438348 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:38.438906 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:38.439358 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:38.439362 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:38.439364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:38.439512 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:38.440245 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:38.440248 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:38.440359 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 12:59:38.440462 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 12:59:38.440651 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":3}
I20260809 12:59:38.441475 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.426578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51562 (local address 127.21.235.125:44077)
0809 12:59:38.427178 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:38.427184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:38.427208 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:59:38.427248 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:38.427254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:38.427436 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:38.427592 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:38.428307 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.429360 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.430525 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.431476 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.431623 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:38.434909 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:38.434919 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:38.434931 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:38.434964 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:38.438193 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.439068 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.439079 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.439085 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.439230 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.439632 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.439640 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.439646 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.440119 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:38.440345 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:38.440656 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 12:59:38.440942 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":464,"thread_start_us":274,"threads_started":1}
I20260809 12:59:38.443950 5362 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:59:38.445111 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.444628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59724)
0809 12:59:38.444821 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.444970 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 12:59:38.445358 5309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
I20260809 12:59:38.446990 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.446631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59728)
0809 12:59:38.446804 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.446878 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 12:59:38.447219 5362 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:36983: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.124.keytab.
Entry for principal kudu/127.21.235.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.124.keytab.
I20260809 12:59:38.460035 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.124:36983
--webserver_interface=127.21.235.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.124.keytab
--principal=kudu/127.21.235.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:40011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:35917,127.21.235.125:44077,127.21.235.124:36983
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:59:38.496544 5362 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } attempt: 1
I20260809 12:59:38.497733 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.497251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59736)
0809 12:59:38.497465 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.497579 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 12:59:38.498888 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.498540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59750)
0809 12:59:38.498711 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.498778 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 12:59:38.499117 5362 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:36983: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
I20260809 12:59:38.559094 5284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } attempt: 5
I20260809 12:59:38.567358 5362 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } attempt: 2
Aug 09 12:59:38 dist-test-slave-gp6n krb5kdc[5190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280377, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for kudu/127.21.235.125@KRBTEST.COM
I20260809 12:59:38.568463 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.567951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59762)
0809 12:59:38.568150 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.568250 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 12:59:38.569669 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.569238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59766)
0809 12:59:38.569437 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.569516 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260809 12:59:38.569994 5362 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:36983: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
I20260809 12:59:38.574518 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.560013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51564 (local address 127.21.235.125:44077)
0809 12:59:38.560189 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:38.560193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:38.560213 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:59:38.560412 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:38.560416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:38.560508 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:38.560617 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:38.561597 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.562269 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.564115 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.564566 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.564660 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:38.568722 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:38.568727 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:38.568730 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:38.568788 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:38.571846 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.572562 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.572569 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.572572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.572702 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.573095 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.573101 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.573105 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.573397 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:38.573558 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:38.573935 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 12:59:38.574164 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:59:38.574532 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.559887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44077 (local address 127.0.0.1:51564)
0809 12:59:38.560113 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.560136 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:38.560276 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:38.560684 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:38.560689 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:38.561199 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:38.561439 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:38.561451 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.562462 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.562470 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:38.563959 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:38.563981 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.564735 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.564743 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:38.564985 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:38.565698 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:38.565722 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:38.568488 (+ 2766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:38.572007 (+ 3519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:38.572012 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:38.572017 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:38.572433 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:38.572818 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:38.572823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:38.572827 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:38.572982 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:38.573520 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:38.573525 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:38.573685 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 12:59:38.573835 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 12:59:38.574166 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 12:59:38.575500 5284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:59:38.576671 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.576277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59780)
0809 12:59:38.576464 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.576547 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 12:59:38.577685 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.577343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59784)
0809 12:59:38.577499 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.577577 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 12:59:38.577916 5284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:36983: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
I20260809 12:59:38.629171 5284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } attempt: 1
I20260809 12:59:38.630471 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.629918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59786)
0809 12:59:38.630119 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.630231 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 12:59:38.631575 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.631252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59788)
0809 12:59:38.631409 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.631476 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 12:59:38.631809 5284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:36983: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
I20260809 12:59:38.661226 5362 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } attempt: 3
I20260809 12:59:38.662613 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.662059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59792)
0809 12:59:38.662269 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.662377 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 12:59:38.663781 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.663363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59808)
0809 12:59:38.663604 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.663670 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
W20260809 12:59:38.664057 5362 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:36983: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
I20260809 12:59:38.700044 5284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } attempt: 2
I20260809 12:59:38.701273 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.700758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59814)
0809 12:59:38.701021 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.701111 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 12:59:38.702344 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.702006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59820)
0809 12:59:38.702178 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.702243 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 12:59:38.702654 5284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:36983: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
W20260809 12:59:38.805874 5371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:38.806205 5371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:38.806444 5371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 12:59:38.807322 5362 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } attempt: 4
I20260809 12:59:38.808571 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.808051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59836)
0809 12:59:38.808280 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.808373 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 12:59:38.808899 5284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } attempt: 3
I20260809 12:59:38.809690 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.809294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59840)
0809 12:59:38.809466 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.809559 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 12:59:38.810005 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.809499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59842)
0809 12:59:38.809672 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.809794 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 12:59:38.809958 5362 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:36983: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
I20260809 12:59:38.811025 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.810649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59846)
0809 12:59:38.810837 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.810928 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260809 12:59:38.811220 5284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:36983: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36983: connect: Connection refused (error 111)
W20260809 12:59:38.814798 5371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:59:38.814931 5371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:38.815040 5371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:38.815147 5371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:59:38.815213 5371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:59:38.825102 5371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:40011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:35917,127.21.235.125:44077,127.21.235.124:36983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.124.keytab
--principal=kudu/127.21.235.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.124:36983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.235.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:38.826529 5371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:38.827869 5371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:38.839764 5377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:38.839735 5379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:38.840556 5376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:38.840535 5371 server_base.cc:1034] running on GCE node
I20260809 12:59:38.841403 5371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:38.842893 5371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:38.844127 5371 hybrid_clock.cc:648] HybridClock initialized: now 1786280378844095 us; error 45 us; skew 500 ppm
I20260809 12:59:38.844501 5371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:38 dist-test-slave-gp6n krb5kdc[5190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280378, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:38.847844 5371 init.cc:377] Logged in from keytab as kudu/127.21.235.124@KRBTEST.COM (short username kudu)
I20260809 12:59:38.849380 5371 webserver.cc:460] Webserver started at http://127.21.235.124:46423/ using document root <none> and password file <none>
I20260809 12:59:38.850116 5371 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:38.850272 5371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:38.850656 5371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:59:38.853757 5371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-2/data/instance:
uuid: "6953fd8959114942bb7ad5dc1f3f9838"
format_stamp: "Formatted at 2026-08-09 12:59:38 on dist-test-slave-gp6n"
server_key: "efd38d637a7a934b9f3fbfcaf0643840"
server_key_iv: "bd3d31edbf8f16981b4216e8db68ec2e"
server_key_version: "encryptionkey@0"
I20260809 12:59:38.854717 5371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-2/wal/instance:
uuid: "6953fd8959114942bb7ad5dc1f3f9838"
format_stamp: "Formatted at 2026-08-09 12:59:38 on dist-test-slave-gp6n"
server_key: "efd38d637a7a934b9f3fbfcaf0643840"
server_key_iv: "bd3d31edbf8f16981b4216e8db68ec2e"
server_key_version: "encryptionkey@0"
I20260809 12:59:38.861477 5371 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260809 12:59:38.865846 5390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:38.867339 5371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:59:38.867552 5371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-2/wal
uuid: "6953fd8959114942bb7ad5dc1f3f9838"
format_stamp: "Formatted at 2026-08-09 12:59:38 on dist-test-slave-gp6n"
server_key: "efd38d637a7a934b9f3fbfcaf0643840"
server_key_iv: "bd3d31edbf8f16981b4216e8db68ec2e"
server_key_version: "encryptionkey@0"
I20260809 12:59:38.867769 5371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:38.881088 5371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:38.885460 5371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:38.885766 5371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:38.904492 5371 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.124:36983
I20260809 12:59:38.904529 5446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.124:36983 every 8 connection(s)
I20260809 12:59:38.905897 5371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-2/data/info.pb
I20260809 12:59:38.907099 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 5371
I20260809 12:59:38.907322 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280163688078-22445-0/minicluster-data/master-2/wal/instance
I20260809 12:59:38.907833 22445 external_mini_cluster.cc:1428] Setting key c5f9a7495050b961b51595e0da4e126a
I20260809 12:59:38.910359 5447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:59:38.919812 5447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:59:38 dist-test-slave-gp6n krb5kdc[5190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.124@KRBTEST.COM
Aug 09 12:59:38 dist-test-slave-gp6n krb5kdc[5190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280378, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.124@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:59:38.934079 5450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.909758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59862 (local address 127.21.235.124:36983)
0809 12:59:38.910591 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:38.910608 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:38.910655 (+ 47us) server_negotiation.cc:407] Connection header received
0809 12:59:38.913042 (+ 2387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:38.913069 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:38.913583 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:38.914026 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:38.915476 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.917563 (+ 2087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.919768 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.920976 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.921186 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:38.926100 (+ 4914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:38.926111 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:38.926124 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:38.926169 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:38.929748 (+ 3579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.930597 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.930603 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.930609 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.930786 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.931192 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.931197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.931199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.931743 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:38.932081 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:38.932540 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 12:59:38.932940 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":326,"threads_started":1}
I20260809 12:59:38.938712 5452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.923188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:35917 (local address 127.0.0.1:38668)
0809 12:59:38.924099 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.924153 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:38.924329 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:38.924737 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:38.924744 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:38.925436 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:38.925702 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:38.925711 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.927160 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.927165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:38.928479 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:38.928491 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.929476 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.929481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:38.929600 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:38.930141 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:38.930158 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:38.932241 (+ 2083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:38.935869 (+ 3628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:38.935879 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:38.935886 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:38.936425 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:38.936858 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:38.936863 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:38.936866 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:38.937013 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:38.937747 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:38.937756 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:38.937978 (+ 222us) client_negotiation.cc:769] Sending connection context
0809 12:59:38.938124 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 12:59:38.938350 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":478,"threads_started":1}
I20260809 12:59:38.939160 5451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.923141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38668 (local address 127.21.235.126:35917)
0809 12:59:38.923732 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:38.923738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:38.924261 (+ 523us) server_negotiation.cc:407] Connection header received
0809 12:59:38.924456 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:38.924459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:38.924544 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:38.924672 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:38.925861 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.926994 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.928629 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.929285 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.929434 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:38.932429 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:38.932436 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:38.932442 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:38.932479 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:38.935729 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.936546 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.936552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.936555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.936735 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.937128 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.937134 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.937137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.937580 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:38.937738 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:38.938304 (+ 566us) server_negotiation.cc:299] Negotiation successful
0809 12:59:38.938641 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":254,"threads_started":1}
I20260809 12:59:38.940704 5447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:59:38 dist-test-slave-gp6n krb5kdc[5190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
Aug 09 12:59:38 dist-test-slave-gp6n krb5kdc[5190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280378, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.124@KRBTEST.COM for kudu/127.21.235.125@KRBTEST.COM
I20260809 12:59:38.954010 5452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.941477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44077 (local address 127.0.0.1:51578)
0809 12:59:38.941635 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.941652 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:38.941741 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:38.942036 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:38.942039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:38.942350 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:38.942581 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:38.942590 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.943573 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.943581 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:38.944438 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:38.944448 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.945348 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.945354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:38.945573 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:38.946117 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:38.946134 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:38.948452 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:38.951583 (+ 3131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:38.951589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:38.951593 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:38.951956 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:38.952355 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:38.952360 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:38.952364 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:38.952511 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:38.953054 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:38.953059 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:38.953217 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 12:59:38.953359 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 12:59:38.953654 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 12:59:38.954062 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.941554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51578 (local address 127.21.235.125:44077)
0809 12:59:38.941737 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:38.941743 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:38.941761 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:59:38.941817 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:38.941823 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:38.941897 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:38.942013 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:38.942730 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.943402 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.944590 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.945185 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.945336 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:38.948650 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:38.948654 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:38.948657 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:38.948692 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:38.951441 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.952091 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.952102 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.952105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.952242 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.952624 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.952630 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.952633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.952924 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:38.953091 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:38.953448 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 12:59:38.953673 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:59:38.954067 5453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.937529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38672 (local address 127.21.235.126:35917)
0809 12:59:38.939388 (+ 1859us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:38.939393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:38.939413 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:59:38.939457 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:38.939460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:38.939520 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:38.939640 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:38.941177 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.942324 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.943627 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.944056 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.944194 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:38.948495 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:38.948501 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:38.948504 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:38.948535 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:38.951359 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.952028 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.952034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.952037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.952170 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.952560 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.952565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.952568 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.952844 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:38.953120 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:38.953415 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 12:59:38.953647 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1697,"thread_start_us":1525,"threads_started":1}
I20260809 12:59:38.955284 5447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:59:38 dist-test-slave-gp6n krb5kdc[5190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280378, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.124@KRBTEST.COM for kudu/127.21.235.124@KRBTEST.COM
Aug 09 12:59:38 dist-test-slave-gp6n krb5kdc[5190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.125@KRBTEST.COM
I20260809 12:59:38.969053 5452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.956033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59868)
0809 12:59:38.956186 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.956206 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:38.956299 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:38.956704 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:38.956707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:38.957084 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:38.957211 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:38.957219 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.958154 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.958159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:38.959076 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:38.959087 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.959777 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.959783 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:38.959958 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:38.960568 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:38.960593 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:38.963032 (+ 2439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:38.966668 (+ 3636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:38.966671 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:38.966674 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:38.967038 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:38.967423 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:38.967429 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:38.967433 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:38.967561 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:38.968042 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:38.968047 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:38.968178 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 12:59:38.968323 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 12:59:38.968615 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 12:59:38.969130 5450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.956352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59868 (local address 127.21.235.124:36983)
0809 12:59:38.956490 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:38.956493 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:38.956506 (+ 13us) server_negotiation.cc:407] Connection header received
0809 12:59:38.956540 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:38.956543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:38.956588 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:38.956687 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:38.957342 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.958016 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.959210 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.959626 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.959762 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:38.963212 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:38.963250 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:38.963255 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:38.963293 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:38.966543 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.967156 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.967163 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.967167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.967318 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.967663 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.967671 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.967674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.967928 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:38.968077 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:38.968397 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 12:59:38.968782 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":143,"server-negotiator.queue_time_us":55}
I20260809 12:59:38.971062 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.955956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51580 (local address 127.21.235.125:44077)
0809 12:59:38.956128 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:38.956134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:38.956187 (+ 53us) server_negotiation.cc:407] Connection header received
0809 12:59:38.956386 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:38.956392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:38.956466 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:38.956567 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:38.957734 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.958473 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.959770 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.960254 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.960380 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:38.965348 (+ 4968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:38.965354 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:38.965358 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:38.965398 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:38.968229 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.968893 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.968899 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.968903 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.969046 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.969519 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.969522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.969524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.969904 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:38.970168 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:38.970481 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 12:59:38.970723 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:59:38.975255 5447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838: Bootstrap starting.
I20260809 12:59:38.975775 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:59:38.978555 5447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838: Neither blocks nor log segments found. Creating new log.
I20260809 12:59:38.979797 5447 log.cc:824] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:38.983275 5447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838: No bootstrap required, opened a new log
I20260809 12:59:38.988415 5284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } attempt: 4
I20260809 12:59:38.994614 5447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } }
I20260809 12:59:38.995229 5447 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:59:38.995416 5447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6953fd8959114942bb7ad5dc1f3f9838, State: Initialized, Role: FOLLOWER
I20260809 12:59:38.996207 5447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } }
I20260809 12:59:39.002589 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.980973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51590 (local address 127.21.235.125:44077)
0809 12:59:38.981172 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:38.981179 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:38.981679 (+ 500us) server_negotiation.cc:407] Connection header received
0809 12:59:38.981968 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:38.981973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:38.982049 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:38.982124 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:38.984545 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.986065 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.988842 (+ 2777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.989749 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:38.989876 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:38.994559 (+ 4683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:38.994569 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:38.994573 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:38.994628 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:38.998815 (+ 4187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:38.999735 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:38.999741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:38.999744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:38.999905 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:39.000472 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:39.000479 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:39.000484 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:39.000844 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:39.001093 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:39.001724 (+ 631us) server_negotiation.cc:299] Negotiation successful
0809 12:59:39.002047 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:59:39.003378 5464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } } }
I20260809 12:59:39.003818 5464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:59:39.005371 5447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:59:39.014922 5453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.980571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38678 (local address 127.21.235.126:35917)
0809 12:59:38.980788 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:38.980796 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:38.986437 (+ 5641us) server_negotiation.cc:407] Connection header received
0809 12:59:38.988314 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:38.988321 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:38.994337 (+ 6016us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:38.995618 (+ 1281us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:38.996530 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.997821 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:39.000763 (+ 2942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:39.001650 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:39.001794 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:39.006138 (+ 4344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:39.006147 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:39.006154 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:39.006199 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:39.010981 (+ 4782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:39.011845 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:39.011851 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:39.011855 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:39.012029 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:39.012522 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:39.012527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:39.012529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:39.012927 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:39.013251 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:39.014266 (+ 1015us) server_negotiation.cc:299] Negotiation successful
0809 12:59:39.014551 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":71}
I20260809 12:59:39.016002 5450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.982266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59870 (local address 127.21.235.124:36983)
0809 12:59:38.982501 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:38.982507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:38.993298 (+ 10791us) server_negotiation.cc:407] Connection header received
0809 12:59:38.993807 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:38.993814 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:38.993890 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:38.993964 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:38.998253 (+ 4289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:38.999373 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:39.003524 (+ 4151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:39.004316 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:39.004457 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:39.008527 (+ 4070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:39.008538 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:39.008545 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:39.008597 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:39.012529 (+ 3932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:39.013309 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:39.013315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:39.013318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:39.013498 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:39.013986 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:39.013991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:39.013994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:39.014533 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:39.014802 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:39.015367 (+ 565us) server_negotiation.cc:299] Negotiation successful
0809 12:59:39.015607 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 12:59:39 dist-test-slave-gp6n krb5kdc[5190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280377, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for kudu/127.21.235.124@KRBTEST.COM
I20260809 12:59:39.026705 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.988967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59878)
0809 12:59:38.989365 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:38.989383 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:38.989483 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:39.008501 (+ 19018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:39.008506 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:39.009109 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:39.009434 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:39.009448 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:39.011925 (+ 2477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:39.011934 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:39.013917 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:39.013938 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:39.015372 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:39.015382 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:39.015651 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:39.016365 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:39.016396 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:39.019477 (+ 3081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:39.023767 (+ 4290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:39.023775 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:39.023779 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:39.024322 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:39.024794 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:39.024798 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:39.024801 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:39.025015 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:39.025684 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:39.025689 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:39.025849 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 12:59:39.026005 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 12:59:39.026279 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263}
I20260809 12:59:39.026773 5463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:38.989671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59878 (local address 127.21.235.124:36983)
0809 12:59:39.007925 (+ 18254us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:39.007933 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:39.007969 (+ 36us) server_negotiation.cc:407] Connection header received
0809 12:59:39.008022 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:39.008027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:39.008099 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:39.009616 (+ 1517us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:39.010030 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:39.011718 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:39.014196 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:39.015156 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:39.016064 (+ 908us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:39.019695 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:39.019700 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:39.019703 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:39.019734 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:39.023615 (+ 3881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:39.024482 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:39.024487 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:39.024490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:39.024647 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:39.025162 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:39.025166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:39.025169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:39.025520 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:39.025718 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:39.026100 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 12:59:39.026299 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18098,"thread_start_us":18535,"threads_started":1}
I20260809 12:59:39.031896 5284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f: Bootstrap starting.
I20260809 12:59:39.035971 5284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f: Neither blocks nor log segments found. Creating new log.
W20260809 12:59:39.037303 5476 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 12:59:39.037495 5476 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 12:59:39.037420 5284 log.cc:824] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:39.041541 5284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f: No bootstrap required, opened a new log
I20260809 12:59:39.025920 5371 master_runner.cc:186] Error getting master registration for 127.21.235.126:35917: OK, instance_id { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" instance_seqno: 1786280377851135 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 12:59:39.041960 5371 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 12:59:39.048647 5284 raft_consensus.cc:348] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } }
I20260809 12:59:39.048969 5284 raft_consensus.cc:374] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:59:39.049096 5284 raft_consensus.cc:729] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e61d7fbe0129421ea5e64ecbea21c43f, State: Initialized, Role: FOLLOWER
I20260809 12:59:39.049705 5284 consensus_queue.cc:260] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } }
I20260809 12:59:39.051337 5479 sys_catalog.cc:455] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } } }
I20260809 12:59:39.051803 5479 sys_catalog.cc:458] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:59:39.052558 5284 sys_catalog.cc:564] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:59:39.054054 5464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:59:39.054337 5464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } }
I20260809 12:59:39.055922 5464 leader_election.cc:288] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e61d7fbe0129421ea5e64ecbea21c43f (127.21.235.126:35917), 2dbb7244f0bb4224a3981b6c2243ed13 (127.21.235.125:44077)
I20260809 12:59:39.056702 5259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6953fd8959114942bb7ad5dc1f3f9838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" is_pre_election: true
I20260809 12:59:39.057421 5259 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6953fd8959114942bb7ad5dc1f3f9838 in term 0.
I20260809 12:59:39.058063 5395 leader_election.cc:302] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6953fd8959114942bb7ad5dc1f3f9838, e61d7fbe0129421ea5e64ecbea21c43f; no voters:
I20260809 12:59:39.058487 5464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:59:39.058630 5464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:59:39.058737 5464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:59:39.062111 5464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } }
I20260809 12:59:39.063818 5464 leader_election.cc:288] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [CANDIDATE]: Term 1 election: Requested vote from peers e61d7fbe0129421ea5e64ecbea21c43f (127.21.235.126:35917), 2dbb7244f0bb4224a3981b6c2243ed13 (127.21.235.125:44077)
I20260809 12:59:39.064561 5259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6953fd8959114942bb7ad5dc1f3f9838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e61d7fbe0129421ea5e64ecbea21c43f"
I20260809 12:59:39.064812 5259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:59:39.068518 5259 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6953fd8959114942bb7ad5dc1f3f9838 in term 1.
I20260809 12:59:39.069010 5395 leader_election.cc:302] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6953fd8959114942bb7ad5dc1f3f9838, e61d7fbe0129421ea5e64ecbea21c43f; no voters:
I20260809 12:59:39.069329 5464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:59:39.069957 5464 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [term 1 LEADER]: Becoming Leader. State: Replica: 6953fd8959114942bb7ad5dc1f3f9838, State: Running, Role: LEADER
I20260809 12:59:39.070586 5464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } }
I20260809 12:59:39.076118 5499 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 12:59:39.076241 5498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6953fd8959114942bb7ad5dc1f3f9838. Latest consensus state: current_term: 1 leader_uuid: "6953fd8959114942bb7ad5dc1f3f9838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } } }
I20260809 12:59:39.076646 5498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [sys.catalog]: This master's current role is: LEADER
I20260809 12:59:39.078334 5501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:59:39.081065 5501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:59:39.080104 5214 master_runner.cc:186] Error getting master registration for 127.21.235.125:44077: OK, instance_id { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" instance_seqno: 1786280378379165 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 12:59:39.090472 5214 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 12:59:39.090651 5236 tablet.cc:2369] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:59:39.093055 5259 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f [term 1 FOLLOWER]: Refusing update from remote peer 6953fd8959114942bb7ad5dc1f3f9838: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:59:39.094154 5464 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:59:39.109201 5362 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } attempt: 5
I20260809 12:59:39.112438 5479 sys_catalog.cc:455] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6953fd8959114942bb7ad5dc1f3f9838. Latest consensus state: current_term: 1 leader_uuid: "6953fd8959114942bb7ad5dc1f3f9838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } } }
I20260809 12:59:39.112787 5479 sys_catalog.cc:458] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:59:39.115331 5503 mvcc.cc:205] Tried to move back new op lower bound from 7316604432752115712 to 7316604432678391808. Current Snapshot: MvccSnapshot[applied={T|T < 7316604432752115712}]
I20260809 12:59:39.119024 5504 mvcc.cc:205] Tried to move back new op lower bound from 7316604432752115712 to 7316604432678391808. Current Snapshot: MvccSnapshot[applied={T|T < 7316604432752115712}]
Aug 09 12:59:39 dist-test-slave-gp6n krb5kdc[5190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280378, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.125@KRBTEST.COM for kudu/127.21.235.124@KRBTEST.COM
I20260809 12:59:39.131461 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:39.109820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36983 (local address 127.0.0.1:59888)
0809 12:59:39.110074 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:39.110103 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:39.110254 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:39.110910 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:39.110916 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:39.111489 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:39.111716 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:39.111729 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:39.115381 (+ 3652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:39.115387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:39.116875 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:39.116892 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:39.118345 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:39.118354 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:39.118610 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:39.119486 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:39.119510 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:39.122676 (+ 3166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:39.128245 (+ 5569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:39.128252 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:39.128257 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:39.128783 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:39.129384 (+ 601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:39.129389 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:39.129393 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:39.129587 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:39.130342 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:39.130347 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:39.130527 (+ 180us) client_negotiation.cc:769] Sending connection context
0809 12:59:39.130682 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 12:59:39.131007 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 12:59:39.132534 5463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:39.110164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59888 (local address 127.21.235.124:36983)
0809 12:59:39.110364 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:39.110372 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:39.110395 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:59:39.110487 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:39.110493 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:39.110584 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:39.110678 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:39.113926 (+ 3248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:39.115168 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:39.117234 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:39.117918 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:39.118063 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:39.122896 (+ 4833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:39.122904 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:39.122909 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:39.122960 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:39.128072 (+ 5112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:39.128927 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:39.128944 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:39.129079 (+ 135us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:39.129225 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:39.129715 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:39.129720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:39.129724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:39.130101 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:39.130265 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:39.131018 (+ 753us) server_negotiation.cc:299] Negotiation successful
0809 12:59:39.131560 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 12:59:39.132769 5479 sys_catalog.cc:455] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6953fd8959114942bb7ad5dc1f3f9838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } } }
I20260809 12:59:39.133051 5479 sys_catalog.cc:458] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:59:39.140502 5362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13: Bootstrap starting.
I20260809 12:59:39.144882 5362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13: Neither blocks nor log segments found. Creating new log.
I20260809 12:59:39.146984 5464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6953fd8959114942bb7ad5dc1f3f9838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } } }
I20260809 12:59:39.147341 5464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [sys.catalog]: This master's current role is: LEADER
I20260809 12:59:39.148703 5501 catalog_manager.cc:1347] Generated new cluster ID: 900f63eb3864403da3f0a835c83a3d80
I20260809 12:59:39.148804 5501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:59:39.149477 5362 log.cc:824] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:39.154644 5362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13: No bootstrap required, opened a new log
I20260809 12:59:39.172660 5362 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } }
I20260809 12:59:39.173179 5362 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:59:39.173333 5362 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dbb7244f0bb4224a3981b6c2243ed13, State: Initialized, Role: FOLLOWER
I20260809 12:59:39.174095 5501 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:59:39.174063 5362 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } }
I20260809 12:59:39.176008 5501 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:59:39.181210 5510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } } }
I20260809 12:59:39.183399 5362 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:59:39.183372 5510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:59:39.183974 5336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6953fd8959114942bb7ad5dc1f3f9838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" is_pre_election: true
I20260809 12:59:39.184579 5336 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6953fd8959114942bb7ad5dc1f3f9838 in term 0.
I20260809 12:59:39.185554 5335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6953fd8959114942bb7ad5dc1f3f9838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dbb7244f0bb4224a3981b6c2243ed13"
I20260809 12:59:39.185879 5335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:59:39.187328 5501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838: Generated new TSK 0
I20260809 12:59:39.188256 5501 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:59:39.190816 5335 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6953fd8959114942bb7ad5dc1f3f9838 in term 1.
I20260809 12:59:39.191581 5334 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13 [term 1 FOLLOWER]: Refusing update from remote peer 6953fd8959114942bb7ad5dc1f3f9838: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:59:39.192592 5498 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:59:39.244410 5510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6953fd8959114942bb7ad5dc1f3f9838. Latest consensus state: current_term: 1 leader_uuid: "6953fd8959114942bb7ad5dc1f3f9838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } } }
I20260809 12:59:39.244410 5518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6953fd8959114942bb7ad5dc1f3f9838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } } }
I20260809 12:59:39.244877 5510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:59:39.244890 5518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:59:39.245806 5530 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 12:59:39.725944 5498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6953fd8959114942bb7ad5dc1f3f9838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61d7fbe0129421ea5e64ecbea21c43f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35917 } } peers { permanent_uuid: "2dbb7244f0bb4224a3981b6c2243ed13" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44077 } } peers { permanent_uuid: "6953fd8959114942bb7ad5dc1f3f9838" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36983 } } }
I20260809 12:59:39.726235 5498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 [sys.catalog]: This master's current role is: LEADER
I20260809 12:59:40.081575 5499 catalog_manager.cc:1259] Loaded cluster ID: 900f63eb3864403da3f0a835c83a3d80
I20260809 12:59:40.081713 5499 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f: loading cluster ID for follower catalog manager: success
I20260809 12:59:40.085600 5499 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f: acquiring CA information for follower catalog manager: success
I20260809 12:59:40.088155 5499 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e61d7fbe0129421ea5e64ecbea21c43f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 12:59:40.251329 5530 catalog_manager.cc:1259] Loaded cluster ID: 900f63eb3864403da3f0a835c83a3d80
I20260809 12:59:40.251457 5530 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13: loading cluster ID for follower catalog manager: success
I20260809 12:59:40.255155 5530 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13: acquiring CA information for follower catalog manager: success
I20260809 12:59:40.257903 5530 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2dbb7244f0bb4224a3981b6c2243ed13: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 12:59:40.310313 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 5214
I20260809 12:59:40.312052 5279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:40.455906 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 5214
I20260809 12:59:40.478999 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 5291
I20260809 12:59:40.480216 5357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:40.543987 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:40.543202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:35917 (local address 127.0.0.1:38684)
0809 12:59:40.543695 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:40.543814 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:35917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":272,"threads_started":1}
W20260809 12:59:40.544281 5395 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.235.126:35917: connect: Connection refused (error 111)
I20260809 12:59:40.547475 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:40.547123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:35917 (local address 127.0.0.1:38686)
0809 12:59:40.547272 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:40.547351 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:35917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 12:59:40.547670 5395 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6953fd8959114942bb7ad5dc1f3f9838 -> Peer e61d7fbe0129421ea5e64ecbea21c43f (127.21.235.126:35917): Couldn't send request to peer e61d7fbe0129421ea5e64ecbea21c43f. Status: Network error: Client connection negotiation failed: client connection to 127.21.235.126:35917: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 12:59:40.585749 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 5291
I20260809 12:59:40.607321 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 5371
I20260809 12:59:40.608475 5441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:40.718498 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 5371
2026-08-09T12:59:40Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4746 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:59:40 dist-test-slave-gp6n krb5kdc[5544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:59:40 dist-test-slave-gp6n krb5kdc[5544](info): set up 2 sockets
Aug 09 12:59:40 dist-test-slave-gp6n krb5kdc[5544](info): commencing operation
krb5kdc: starting...
W20260809 12:59:41.899902 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.007s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:59:42 dist-test-slave-gp6n krb5kdc[5544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:59:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:59:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 12:59:42.182054 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:41999
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:40427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:41999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:59:42.539626 5558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:42.539911 5558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:42.540081 5558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:42.547330 5558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:59:42.547477 5558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:42.547591 5558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:42.547681 5558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:59:42.547762 5558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:59:42.557724 5558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:40427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:41999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:41999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:42.559115 5558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:42.560475 5558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:42.572788 5564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:42.572757 5565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:42.573216 5568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:42.573621 5558 server_base.cc:1034] running on GCE node
I20260809 12:59:42.574398 5558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:42.575917 5558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:42.577154 5558 hybrid_clock.cc:648] HybridClock initialized: now 1786280382577096 us; error 42 us; skew 500 ppm
I20260809 12:59:42.577508 5558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:42 dist-test-slave-gp6n krb5kdc[5544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280382, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:42.580615 5558 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:59:42.582168 5558 webserver.cc:460] Webserver started at http://127.21.235.126:33609/ using document root <none> and password file <none>
I20260809 12:59:42.582959 5558 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:42.583112 5558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:42.583434 5558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:59:42.586495 5558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "01a3f919eb594ecd8bbfae9c3bf41254"
format_stamp: "Formatted at 2026-08-09 12:59:42 on dist-test-slave-gp6n"
server_key: "0cdb834ad30525be34b99860a5dc6c16"
server_key_iv: "fa5229a9b3f7ec50d6c3106a3714cd4c"
server_key_version: "encryptionkey@0"
I20260809 12:59:42.587338 5558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "01a3f919eb594ecd8bbfae9c3bf41254"
format_stamp: "Formatted at 2026-08-09 12:59:42 on dist-test-slave-gp6n"
server_key: "0cdb834ad30525be34b99860a5dc6c16"
server_key_iv: "fa5229a9b3f7ec50d6c3106a3714cd4c"
server_key_version: "encryptionkey@0"
I20260809 12:59:42.592557 5558 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:59:42.596125 5575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:42.597478 5558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:59:42.597668 5558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "01a3f919eb594ecd8bbfae9c3bf41254"
format_stamp: "Formatted at 2026-08-09 12:59:42 on dist-test-slave-gp6n"
server_key: "0cdb834ad30525be34b99860a5dc6c16"
server_key_iv: "fa5229a9b3f7ec50d6c3106a3714cd4c"
server_key_version: "encryptionkey@0"
I20260809 12:59:42.597882 5558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:42.613133 5558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:42.616964 5558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:42.617256 5558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:42.636039 5558 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:41999
I20260809 12:59:42.636045 5630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:41999 every 8 connection(s)
I20260809 12:59:42.637435 5558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:59:42.637985 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 5558
I20260809 12:59:42.638224 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:59:42.639111 22445 external_mini_cluster.cc:1428] Setting key 26f1a960f92f0f941e93b24a8ff6463c
I20260809 12:59:42.642129 5631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 12:59:42 dist-test-slave-gp6n krb5kdc[5544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:59:42.655292 5631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254: Bootstrap starting.
I20260809 12:59:42.658993 5631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254: Neither blocks nor log segments found. Creating new log.
I20260809 12:59:42.660293 5631 log.cc:824] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:42.663708 5631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254: No bootstrap required, opened a new log
I20260809 12:59:42.664057 5634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:42.641015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48510 (local address 127.21.235.126:41999)
0809 12:59:42.641834 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:42.641849 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:42.641901 (+ 52us) server_negotiation.cc:407] Connection header received
0809 12:59:42.644297 (+ 2396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:42.644324 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:42.644812 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:42.645252 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:42.646733 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:42.648852 (+ 2119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:42.650208 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:42.651277 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:42.651481 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:42.656108 (+ 4627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:42.656116 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:42.656130 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:42.656174 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:42.659431 (+ 3257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:42.660287 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:42.660293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:42.660298 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:42.660530 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:42.661016 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:42.661023 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:42.661029 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:42.661813 (+ 784us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:42.662027 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:42.662615 (+ 588us) server_negotiation.cc:299] Negotiation successful
0809 12:59:42.662993 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":331,"threads_started":1}
I20260809 12:59:42.668279 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:59:42.672884 5631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a3f919eb594ecd8bbfae9c3bf41254" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 41999 } }
I20260809 12:59:42.673288 5631 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:59:42.673418 5631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a3f919eb594ecd8bbfae9c3bf41254, State: Initialized, Role: FOLLOWER
I20260809 12:59:42.674059 5631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a3f919eb594ecd8bbfae9c3bf41254" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 41999 } }
I20260809 12:59:42.674297 5631 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:59:42.674472 5631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:59:42.674656 5631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:59:42.676857 5631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a3f919eb594ecd8bbfae9c3bf41254" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 41999 } }
I20260809 12:59:42.677383 5631 leader_election.cc:302] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01a3f919eb594ecd8bbfae9c3bf41254; no voters:
I20260809 12:59:42.678006 5631 leader_election.cc:288] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:59:42.678143 5637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:59:42.678865 5637 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254 [term 1 LEADER]: Becoming Leader. State: Replica: 01a3f919eb594ecd8bbfae9c3bf41254, State: Running, Role: LEADER
I20260809 12:59:42.679406 5637 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a3f919eb594ecd8bbfae9c3bf41254" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 41999 } }
I20260809 12:59:42.680222 5631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:59:42.682247 5638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01a3f919eb594ecd8bbfae9c3bf41254" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a3f919eb594ecd8bbfae9c3bf41254" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 41999 } } }
I20260809 12:59:42.682703 5638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254 [sys.catalog]: This master's current role is: LEADER
I20260809 12:59:42.682492 5640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01a3f919eb594ecd8bbfae9c3bf41254. Latest consensus state: current_term: 1 leader_uuid: "01a3f919eb594ecd8bbfae9c3bf41254" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a3f919eb594ecd8bbfae9c3bf41254" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 41999 } } }
I20260809 12:59:42.682890 5640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254 [sys.catalog]: This master's current role is: LEADER
I20260809 12:59:42.684790 5645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:59:42.690335 5645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:59:42.698613 5645 catalog_manager.cc:1347] Generated new cluster ID: 8481a4ed297d4712ba77e1fe7acc1e04
I20260809 12:59:42.698717 5645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:59:42.726953 5645 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:59:42.728374 5645 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:59:42.739125 5645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01a3f919eb594ecd8bbfae9c3bf41254: Generated new TSK 0
I20260809 12:59:42.740355 5645 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:59:42.780792 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 5558
I20260809 12:59:42.782042 5625 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:42.879079 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 5558
2026-08-09T12:59:42Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2157 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:59:42 dist-test-slave-gp6n krb5kdc[5664](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:59:42 dist-test-slave-gp6n krb5kdc[5664](info): set up 2 sockets
Aug 09 12:59:42 dist-test-slave-gp6n krb5kdc[5664](info): commencing operation
krb5kdc: starting...
W20260809 12:59:44.056811 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.002s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 12:59:44 dist-test-slave-gp6n krb5kdc[5664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:59:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:59:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 12:59:44.339622 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:40589
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:40589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:59:44.680213 5677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:44.680516 5677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:44.680608 5677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:44.687640 5677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:59:44.687739 5677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:44.687785 5677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:44.687839 5677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:59:44.687887 5677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:59:44.697314 5677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:40589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:40589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:44.698571 5677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:44.699824 5677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:44.711645 5682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:44.711575 5686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:44.711619 5684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:44.712605 5677 server_base.cc:1034] running on GCE node
I20260809 12:59:44.713433 5677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:44.714906 5677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:44.716157 5677 hybrid_clock.cc:648] HybridClock initialized: now 1786280384716109 us; error 51 us; skew 500 ppm
I20260809 12:59:44.716521 5677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:44 dist-test-slave-gp6n krb5kdc[5664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280384, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:44.719940 5677 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:59:44.721379 5677 webserver.cc:460] Webserver started at http://127.21.235.126:43615/ using document root <none> and password file <none>
I20260809 12:59:44.722057 5677 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:44.722203 5677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:44.722561 5677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:59:44.725602 5677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d"
format_stamp: "Formatted at 2026-08-09 12:59:44 on dist-test-slave-gp6n"
server_key: "01f6e554a73bfdbd04ccc1245b17a941"
server_key_iv: "fb8bdf9c3910f4eafe0d61019e067e6b"
server_key_version: "encryptionkey@0"
I20260809 12:59:44.726454 5677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d"
format_stamp: "Formatted at 2026-08-09 12:59:44 on dist-test-slave-gp6n"
server_key: "01f6e554a73bfdbd04ccc1245b17a941"
server_key_iv: "fb8bdf9c3910f4eafe0d61019e067e6b"
server_key_version: "encryptionkey@0"
I20260809 12:59:44.731696 5677 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 12:59:44.735476 5693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:44.736933 5677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:59:44.737126 5677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d"
format_stamp: "Formatted at 2026-08-09 12:59:44 on dist-test-slave-gp6n"
server_key: "01f6e554a73bfdbd04ccc1245b17a941"
server_key_iv: "fb8bdf9c3910f4eafe0d61019e067e6b"
server_key_version: "encryptionkey@0"
I20260809 12:59:44.737336 5677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:44.772270 5677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:44.776685 5677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:44.776989 5677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:44.796526 5677 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:40589
I20260809 12:59:44.796578 5748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:40589 every 8 connection(s)
I20260809 12:59:44.797969 5677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:59:44.801810 5749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:59:44.806685 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 5677
I20260809 12:59:44.806906 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 12:59:44.807601 22445 external_mini_cluster.cc:1428] Setting key 2bdccf7e8d11d7972ee6eb0e713d836b
I20260809 12:59:44.813114 5749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d: Bootstrap starting.
I20260809 12:59:44.816923 5749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d: Neither blocks nor log segments found. Creating new log.
I20260809 12:59:44.818200 5749 log.cc:824] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:44.821859 5749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d: No bootstrap required, opened a new log
Aug 09 12:59:44 dist-test-slave-gp6n krb5kdc[5664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:59:44.832758 5749 raft_consensus.cc:348] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40589 } }
I20260809 12:59:44.833232 5749 raft_consensus.cc:374] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:59:44.833343 5749 raft_consensus.cc:729] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87bf791a9d742a9b1d0bd8e4bb3134d, State: Initialized, Role: FOLLOWER
I20260809 12:59:44.833567 5752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:44.809610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57164 (local address 127.21.235.126:40589)
0809 12:59:44.810540 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:44.810552 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:44.810598 (+ 46us) server_negotiation.cc:407] Connection header received
0809 12:59:44.812348 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:44.812373 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:44.812768 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:44.813092 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:44.815061 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:44.816350 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:44.818329 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:44.819548 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:44.819727 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:44.824603 (+ 4876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:44.824614 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:44.824633 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:44.824687 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:44.828900 (+ 4213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:44.829629 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:44.829642 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:44.829651 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:44.829862 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:44.830206 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:44.830212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:44.830216 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:44.831033 (+ 817us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:44.831373 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:44.831699 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 12:59:44.832146 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":614,"thread_start_us":411,"threads_started":1}
I20260809 12:59:44.834018 5749 consensus_queue.cc:260] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40589 } }
I20260809 12:59:44.834267 5749 raft_consensus.cc:388] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:59:44.834475 5749 raft_consensus.cc:482] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:59:44.834672 5749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:59:44.837016 5749 raft_consensus.cc:504] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40589 } }
I20260809 12:59:44.837509 5749 leader_election.cc:302] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c87bf791a9d742a9b1d0bd8e4bb3134d; no voters:
I20260809 12:59:44.838112 5749 leader_election.cc:288] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:59:44.838261 5754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:59:44.839466 5754 raft_consensus.cc:686] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 1 LEADER]: Becoming Leader. State: Replica: c87bf791a9d742a9b1d0bd8e4bb3134d, State: Running, Role: LEADER
I20260809 12:59:44.840117 5754 consensus_queue.cc:237] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40589 } }
I20260809 12:59:44.840312 5749 sys_catalog.cc:564] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:59:44.843837 5755 sys_catalog.cc:455] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40589 } } }
I20260809 12:59:44.844053 5756 sys_catalog.cc:455] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c87bf791a9d742a9b1d0bd8e4bb3134d. Latest consensus state: current_term: 1 leader_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40589 } } }
I20260809 12:59:44.844313 5755 sys_catalog.cc:458] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [sys.catalog]: This master's current role is: LEADER
I20260809 12:59:44.844375 5756 sys_catalog.cc:458] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [sys.catalog]: This master's current role is: LEADER
I20260809 12:59:44.845757 5763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:59:44.849965 5763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:59:44.859617 5763 catalog_manager.cc:1347] Generated new cluster ID: 5f72890e10f44d78a008594817f0213d
I20260809 12:59:44.859738 5763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:59:44.877415 5763 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:59:44.878835 5763 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:59:44.888166 5763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d: Generated new TSK 0
I20260809 12:59:44.889014 5763 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
I20260809 12:59:44.962946 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40589
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:59:45.350700 5778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:45.351022 5778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:45.351156 5778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:45.358686 5778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:45.358816 5778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:45.359007 5778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:59:45.368834 5778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:45.370172 5778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:45.371567 5778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:45.385139 5784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:45.385211 5785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:45.385289 5787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:45.386423 5778 server_base.cc:1034] running on GCE node
I20260809 12:59:45.387024 5778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:45.388123 5778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:45.389353 5778 hybrid_clock.cc:648] HybridClock initialized: now 1786280385389287 us; error 56 us; skew 500 ppm
I20260809 12:59:45.389700 5778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:45 dist-test-slave-gp6n krb5kdc[5664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280385, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:45.393167 5778 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 12:59:45.394724 5778 webserver.cc:460] Webserver started at http://127.21.235.65:37913/ using document root <none> and password file <none>
I20260809 12:59:45.395458 5778 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:45.395596 5778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:45.395922 5778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:59:45.399020 5778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "8a6444b55f854350a5b10bbc6203df96"
format_stamp: "Formatted at 2026-08-09 12:59:45 on dist-test-slave-gp6n"
server_key: "02452535ac568896337ab293a8fd4862"
server_key_iv: "c10546a5f265bfc24354472c84da4726"
server_key_version: "encryptionkey@0"
I20260809 12:59:45.399852 5778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "8a6444b55f854350a5b10bbc6203df96"
format_stamp: "Formatted at 2026-08-09 12:59:45 on dist-test-slave-gp6n"
server_key: "02452535ac568896337ab293a8fd4862"
server_key_iv: "c10546a5f265bfc24354472c84da4726"
server_key_version: "encryptionkey@0"
I20260809 12:59:45.405282 5778 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:59:45.408938 5794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:45.410374 5778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:59:45.410643 5778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "8a6444b55f854350a5b10bbc6203df96"
format_stamp: "Formatted at 2026-08-09 12:59:45 on dist-test-slave-gp6n"
server_key: "02452535ac568896337ab293a8fd4862"
server_key_iv: "c10546a5f265bfc24354472c84da4726"
server_key_version: "encryptionkey@0"
I20260809 12:59:45.410867 5778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:45.425215 5778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:45.429244 5778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:45.429572 5778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:45.430612 5778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:59:45.431990 5778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:59:45.432096 5778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:45.432245 5778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:59:45.432323 5778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:45.466706 5778 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:42643
I20260809 12:59:45.466743 5919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:42643 every 8 connection(s)
I20260809 12:59:45.468048 5778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:59:45.475917 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 5778
I20260809 12:59:45.476162 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 12:59:45.476914 22445 external_mini_cluster.cc:1428] Setting key 286f0f1f867ca2bc195098b982d76248
Aug 09 12:59:45 dist-test-slave-gp6n krb5kdc[5664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280385, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:59:45.488665 5922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:45.470740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:47525 (local address 127.21.235.126:40589)
0809 12:59:45.471190 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:45.471195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:45.472193 (+ 998us) server_negotiation.cc:407] Connection header received
0809 12:59:45.474024 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:45.474030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:45.474118 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:45.474240 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:45.475802 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:45.476657 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:45.478568 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:45.479119 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:45.479259 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:45.482870 (+ 3611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:45.482875 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:45.482878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:45.482911 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:45.485500 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:45.486238 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:45.486243 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:45.486245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:45.486368 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:45.486864 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:45.486869 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:45.486872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:45.487144 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:45.487304 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:45.488028 (+ 724us) server_negotiation.cc:299] Negotiation successful
0809 12:59:45.488265 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":230,"threads_started":1}
I20260809 12:59:45.489890 5923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:45.471348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:47525)
0809 12:59:45.472040 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:45.472080 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:45.473734 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:45.474527 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:45.474536 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:45.475050 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:45.475621 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:45.475638 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:45.476812 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:45.476822 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:45.478336 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:45.478350 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:45.479369 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:45.479378 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:45.479642 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:45.480357 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:45.480381 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:45.482647 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:45.485645 (+ 2998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:45.485654 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:45.485671 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:45.486100 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:45.486531 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:45.486536 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:45.486540 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:45.486741 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:45.487276 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:45.487284 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:45.487637 (+ 353us) client_negotiation.cc:769] Sending connection context
0809 12:59:45.487901 (+ 264us) client_negotiation.cc:240] Negotiation successful
0809 12:59:45.488475 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":260,"threads_started":1}
I20260809 12:59:45.491744 5920 heartbeater.cc:347] Connected to a master server at 127.21.235.126:40589
I20260809 12:59:45.492074 5920 heartbeater.cc:464] Registering TS with master...
I20260809 12:59:45.492825 5920 heartbeater.cc:511] Master 127.21.235.126:40589 requested a full tablet report, sending...
I20260809 12:59:45.495095 5712 ts_manager.cc:194] Registered new tserver with Master: 8a6444b55f854350a5b10bbc6203df96 (127.21.235.65:42643)
I20260809 12:59:45.496579 5712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:47525
WARNING: no policy specified for kudu/127.21.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
I20260809 12:59:45.535916 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40589
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:59:45.869402 5926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:45.869704 5926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:45.869889 5926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:45.876925 5926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:45.877031 5926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:45.877190 5926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:59:45.886822 5926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:45.888103 5926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:45.889417 5926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:45.902685 5933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:45.902638 5936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:45.902767 5934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:45.903527 5926 server_base.cc:1034] running on GCE node
I20260809 12:59:45.904124 5926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:45.905059 5926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:45.906283 5926 hybrid_clock.cc:648] HybridClock initialized: now 1786280385906231 us; error 60 us; skew 500 ppm
I20260809 12:59:45.906620 5926 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:45 dist-test-slave-gp6n krb5kdc[5664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280385, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:45.909687 5926 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 12:59:45.911188 5926 webserver.cc:460] Webserver started at http://127.21.235.66:46425/ using document root <none> and password file <none>
I20260809 12:59:45.911842 5926 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:45.911995 5926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:45.912290 5926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:59:45.915274 5926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "ddd38535e34b4ffb863ee3126190ab3d"
format_stamp: "Formatted at 2026-08-09 12:59:45 on dist-test-slave-gp6n"
server_key: "f085f1cc929d0e379918826c255c36a8"
server_key_iv: "3ae04146b7d434443e7d91dee4ffab22"
server_key_version: "encryptionkey@0"
I20260809 12:59:45.916088 5926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "ddd38535e34b4ffb863ee3126190ab3d"
format_stamp: "Formatted at 2026-08-09 12:59:45 on dist-test-slave-gp6n"
server_key: "f085f1cc929d0e379918826c255c36a8"
server_key_iv: "3ae04146b7d434443e7d91dee4ffab22"
server_key_version: "encryptionkey@0"
I20260809 12:59:45.921126 5926 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 12:59:45.924500 5943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:45.925838 5926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:59:45.926036 5926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "ddd38535e34b4ffb863ee3126190ab3d"
format_stamp: "Formatted at 2026-08-09 12:59:45 on dist-test-slave-gp6n"
server_key: "f085f1cc929d0e379918826c255c36a8"
server_key_iv: "3ae04146b7d434443e7d91dee4ffab22"
server_key_version: "encryptionkey@0"
I20260809 12:59:45.926246 5926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:45.943630 5926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:45.948669 5926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:45.948975 5926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:45.949905 5926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:59:45.951366 5926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:59:45.951474 5926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:45.951619 5926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:59:45.951670 5926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:45.983083 5926 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:44063
I20260809 12:59:45.983126 6066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:44063 every 8 connection(s)
I20260809 12:59:45.984386 5926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:59:45.991706 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 5926
I20260809 12:59:45.991892 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 12:59:45.992558 22445 external_mini_cluster.cc:1428] Setting key daafdbe6b8b7241db332a8460f761c82
Aug 09 12:59:45 dist-test-slave-gp6n krb5kdc[5664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280385, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:59:46.006354 5922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:45.986981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:36135 (local address 127.21.235.126:40589)
0809 12:59:45.987203 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:45.987208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:45.988313 (+ 1105us) server_negotiation.cc:407] Connection header received
0809 12:59:45.990199 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:45.990205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:45.990302 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:45.990456 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:45.992142 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:45.993241 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:45.995077 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:45.995799 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:45.995947 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:45.999936 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:45.999942 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:45.999945 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:45.999986 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:46.002748 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:46.003685 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:46.003689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:46.003691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:46.003819 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:46.004306 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:46.004309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:46.004311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:46.004564 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:46.004675 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:46.005651 (+ 976us) server_negotiation.cc:299] Negotiation successful
0809 12:59:46.005921 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260809 12:59:46.007475 6069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:45.987428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:36135)
0809 12:59:45.988147 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:45.988189 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:45.989905 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:45.990709 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:45.990718 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:45.991243 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:45.991871 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:45.991892 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:45.993421 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:45.993429 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:45.994917 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:45.994929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:45.996018 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:45.996023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:45.996229 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:45.996922 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:45.996959 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:45.999710 (+ 2751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:46.002885 (+ 3175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:46.002897 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:46.002918 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:46.003561 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:46.003927 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:46.003934 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:46.003940 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:46.004193 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:46.004688 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:46.004699 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:46.005178 (+ 479us) client_negotiation.cc:769] Sending connection context
0809 12:59:46.005520 (+ 342us) client_negotiation.cc:240] Negotiation successful
0809 12:59:46.005926 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":291,"threads_started":1}
I20260809 12:59:46.009192 6067 heartbeater.cc:347] Connected to a master server at 127.21.235.126:40589
I20260809 12:59:46.009580 6067 heartbeater.cc:464] Registering TS with master...
I20260809 12:59:46.010470 6067 heartbeater.cc:511] Master 127.21.235.126:40589 requested a full tablet report, sending...
I20260809 12:59:46.012066 5712 ts_manager.cc:194] Registered new tserver with Master: ddd38535e34b4ffb863ee3126190ab3d (127.21.235.66:44063)
I20260809 12:59:46.013016 5712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:36135
WARNING: no policy specified for kudu/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
I20260809 12:59:46.057392 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40589
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:59:46.403291 6072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:46.403573 6072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:46.403743 6072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:46.411020 6072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:46.411129 6072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:46.411336 6072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:59:46.421171 6072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:46.422489 6072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:46.423804 6072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:46.437156 6078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:46.437129 6079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:46.437408 6081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:46.438479 6072 server_base.cc:1034] running on GCE node
I20260809 12:59:46.439127 6072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:46.440092 6072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:46.441331 6072 hybrid_clock.cc:648] HybridClock initialized: now 1786280386441274 us; error 45 us; skew 500 ppm
I20260809 12:59:46.441795 6072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:46 dist-test-slave-gp6n krb5kdc[5664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280386, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:46.445142 6072 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 12:59:46.446643 6072 webserver.cc:460] Webserver started at http://127.21.235.67:40971/ using document root <none> and password file <none>
I20260809 12:59:46.447337 6072 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:46.447687 6072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:46.448179 6072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:59:46.451678 6072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "b1b83dc844244333baaea9e3cb255212"
format_stamp: "Formatted at 2026-08-09 12:59:46 on dist-test-slave-gp6n"
server_key: "6eab2ef5aa3b3eeaf9629545c6605e66"
server_key_iv: "61b3a1291d8565ee5d7bcc75dc8a3bce"
server_key_version: "encryptionkey@0"
I20260809 12:59:46.452605 6072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "b1b83dc844244333baaea9e3cb255212"
format_stamp: "Formatted at 2026-08-09 12:59:46 on dist-test-slave-gp6n"
server_key: "6eab2ef5aa3b3eeaf9629545c6605e66"
server_key_iv: "61b3a1291d8565ee5d7bcc75dc8a3bce"
server_key_version: "encryptionkey@0"
I20260809 12:59:46.458266 6072 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:59:46.461731 6089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:46.463230 6072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:59:46.463436 6072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "b1b83dc844244333baaea9e3cb255212"
format_stamp: "Formatted at 2026-08-09 12:59:46 on dist-test-slave-gp6n"
server_key: "6eab2ef5aa3b3eeaf9629545c6605e66"
server_key_iv: "61b3a1291d8565ee5d7bcc75dc8a3bce"
server_key_version: "encryptionkey@0"
I20260809 12:59:46.463640 6072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:46.480660 6072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:46.485127 6072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:46.485438 6072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:46.486452 6072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:59:46.487793 6072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:59:46.487901 6072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:46.488050 6072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:59:46.488124 6072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:46.499666 5920 heartbeater.cc:503] Master 127.21.235.126:40589 was elected leader, sending a full tablet report...
I20260809 12:59:46.522338 6072 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:38663
I20260809 12:59:46.522383 6209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:38663 every 8 connection(s)
I20260809 12:59:46.523651 6072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:59:46.524540 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 6072
I20260809 12:59:46.524749 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 12:59:46.525328 22445 external_mini_cluster.cc:1428] Setting key 448104df801114c0d348bf6fec4a744c
Aug 09 12:59:46 dist-test-slave-gp6n krb5kdc[5664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280386, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:59:46.545112 6212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:46.526519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:40319 (local address 127.21.235.126:40589)
0809 12:59:46.527147 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:46.527154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:46.528254 (+ 1100us) server_negotiation.cc:407] Connection header received
0809 12:59:46.530336 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:46.530345 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:46.530484 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:46.530598 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:46.532022 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.533075 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.534645 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.535054 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.535159 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:46.538586 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:46.538591 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:46.538595 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:46.538631 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:46.542305 (+ 3674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:46.543071 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:46.543076 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:46.543078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:46.543211 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:46.543607 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:46.543610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:46.543612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:46.543904 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:46.544018 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:46.544668 (+ 650us) server_negotiation.cc:299] Negotiation successful
0809 12:59:46.544845 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":333,"threads_started":1}
I20260809 12:59:46.546062 6213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:46.527306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:40319)
0809 12:59:46.528107 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:46.528144 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:46.530011 (+ 1867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:46.530798 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:46.530806 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:46.531323 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:46.531871 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:46.531888 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.533221 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.533225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:46.534496 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:46.534509 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.535240 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.535244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:46.535466 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:46.536096 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:46.536122 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:46.538339 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:46.542481 (+ 4142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:46.542489 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:46.542500 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:46.542918 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:46.543307 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:46.543326 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:46.543328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:46.543502 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:46.544007 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:46.544012 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:46.544315 (+ 303us) client_negotiation.cc:769] Sending connection context
0809 12:59:46.544565 (+ 250us) client_negotiation.cc:240] Negotiation successful
0809 12:59:46.544931 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":384,"threads_started":1}
I20260809 12:59:46.547616 6210 heartbeater.cc:347] Connected to a master server at 127.21.235.126:40589
I20260809 12:59:46.547925 6210 heartbeater.cc:464] Registering TS with master...
I20260809 12:59:46.548678 6210 heartbeater.cc:511] Master 127.21.235.126:40589 requested a full tablet report, sending...
I20260809 12:59:46.550240 5712 ts_manager.cc:194] Registered new tserver with Master: b1b83dc844244333baaea9e3cb255212 (127.21.235.67:38663)
I20260809 12:59:46.551147 5712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:40319
I20260809 12:59:46.553974 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:59:46.571441 6212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:46.558649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57180 (local address 127.21.235.126:40589)
0809 12:59:46.558819 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:46.558824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:46.559340 (+ 516us) server_negotiation.cc:407] Connection header received
0809 12:59:46.559460 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:46.559463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:46.559537 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:46.559623 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:46.560943 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.561674 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.563095 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.563589 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.563701 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:46.566047 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:46.566052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:46.566056 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:46.566088 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:46.568901 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:46.569591 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:46.569595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:46.569598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:46.569725 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:46.570054 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:46.570058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:46.570060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:46.570395 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:46.570660 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:46.570978 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 12:59:46.571180 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 12:59:46.575507 5712 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:59:46.583105 5712 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:59:46.612525 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:46.595390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:44063 (local address 127.0.0.1:47522)
0809 12:59:46.596318 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:46.596374 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:46.596556 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:46.598433 (+ 1877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:46.598445 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:46.598490 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:59:46.599052 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:46.599069 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.604452 (+ 5383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.604461 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:46.609139 (+ 4678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:46.609165 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.611392 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.611400 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:46.611795 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:46.611844 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 12:59:46.611934 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 12:59:46.612096 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":88,"thread_start_us":548,"threads_started":1}
I20260809 12:59:46.612542 6225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:46.595614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47522 (local address 127.21.235.66:44063)
0809 12:59:46.597596 (+ 1982us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:46.597608 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:46.597645 (+ 37us) server_negotiation.cc:407] Connection header received
0809 12:59:46.597766 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:46.597777 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:46.598102 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:46.600617 (+ 2515us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:59:46.600924 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.604261 (+ 3337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.609435 (+ 5174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.611148 (+ 1713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.611712 (+ 564us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:46.611952 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:46.612030 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 12:59:46.612220 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1762,"thread_start_us":1622,"threads_started":1}
I20260809 12:59:46.616845 5994 tablet_service.cc:1467] Processing CreateTablet for tablet cab3a694eecb4125bfd125ccbf90559d (DEFAULT_TABLE table=test-table [id=94511486c6f7417592fe2fe98dc4081d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:59:46.618239 5994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cab3a694eecb4125bfd125ccbf90559d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:59:46.618779 6227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:46.595805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:42643 (local address 127.0.0.1:54518)
0809 12:59:46.608702 (+ 12897us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:46.608726 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:46.608869 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:46.611557 (+ 2688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:46.611566 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:46.611592 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:59:46.612131 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:46.612144 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.614205 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.614210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:46.616507 (+ 2297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:46.616522 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.617870 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.617877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:46.618050 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:46.618083 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 12:59:46.618178 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 12:59:46.618335 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12727,"mutex_wait_us":73,"thread_start_us":13166,"threads_started":1}
I20260809 12:59:46.619635 6228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:46.596103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54518 (local address 127.21.235.65:42643)
0809 12:59:46.610704 (+ 14601us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:46.610712 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:46.610739 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:59:46.610842 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:46.610852 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:46.611083 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:46.611295 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:59:46.612337 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.614046 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.616683 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.617673 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.617781 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:46.617949 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:46.619137 (+ 1188us) server_negotiation.cc:299] Negotiation successful
0809 12:59:46.619317 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14443,"thread_start_us":14293,"threads_started":1}
I20260809 12:59:46.620417 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:46.596254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:38663 (local address 127.0.0.1:50406)
0809 12:59:46.610347 (+ 14093us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:46.610370 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:46.610517 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:46.611126 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:46.611131 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:46.611155 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:59:46.611581 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:46.611591 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.614820 (+ 3229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.614828 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:46.616520 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:46.616532 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.619595 (+ 3063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.619601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:46.619769 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:46.619811 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 12:59:46.619933 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 12:59:46.620065 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13937,"mutex_wait_us":35,"thread_start_us":13738,"threads_started":1}
I20260809 12:59:46.621093 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:46.596699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50406 (local address 127.21.235.67:38663)
0809 12:59:46.609794 (+ 13095us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:46.609806 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:46.610637 (+ 831us) server_negotiation.cc:407] Connection header received
0809 12:59:46.610731 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:46.610737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:46.610936 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:46.611139 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:59:46.613163 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.614685 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.618022 (+ 3337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.619399 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.619542 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:46.619770 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:46.620726 (+ 956us) server_negotiation.cc:299] Negotiation successful
0809 12:59:46.620856 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12881,"thread_start_us":12727,"threads_started":1}
I20260809 12:59:46.623585 5849 tablet_service.cc:1467] Processing CreateTablet for tablet cab3a694eecb4125bfd125ccbf90559d (DEFAULT_TABLE table=test-table [id=94511486c6f7417592fe2fe98dc4081d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:59:46.623821 6140 tablet_service.cc:1467] Processing CreateTablet for tablet cab3a694eecb4125bfd125ccbf90559d (DEFAULT_TABLE table=test-table [id=94511486c6f7417592fe2fe98dc4081d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:59:46.624998 6140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cab3a694eecb4125bfd125ccbf90559d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:59:46.624997 5849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cab3a694eecb4125bfd125ccbf90559d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:59:46.633409 6231 tablet_bootstrap.cc:492] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d: Bootstrap starting.
I20260809 12:59:46.637205 6232 tablet_bootstrap.cc:492] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212: Bootstrap starting.
I20260809 12:59:46.638594 6231 tablet_bootstrap.cc:654] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d: Neither blocks nor log segments found. Creating new log.
I20260809 12:59:46.639138 6233 tablet_bootstrap.cc:492] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96: Bootstrap starting.
I20260809 12:59:46.640122 6231 log.cc:824] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:46.640992 6232 tablet_bootstrap.cc:654] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212: Neither blocks nor log segments found. Creating new log.
I20260809 12:59:46.642226 6232 log.cc:824] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:46.642886 6233 tablet_bootstrap.cc:654] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96: Neither blocks nor log segments found. Creating new log.
I20260809 12:59:46.643596 6231 tablet_bootstrap.cc:492] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d: No bootstrap required, opened a new log
I20260809 12:59:46.643868 6231 ts_tablet_manager.cc:1397] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260809 12:59:46.644091 6233 log.cc:824] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:46.645025 6232 tablet_bootstrap.cc:492] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212: No bootstrap required, opened a new log
I20260809 12:59:46.645298 6232 ts_tablet_manager.cc:1397] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260809 12:59:46.648803 6233 tablet_bootstrap.cc:492] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96: No bootstrap required, opened a new log
I20260809 12:59:46.649061 6233 ts_tablet_manager.cc:1397] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 12:59:46.652995 6232 raft_consensus.cc:348] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } }
I20260809 12:59:46.653391 6232 raft_consensus.cc:374] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:59:46.653499 6232 raft_consensus.cc:729] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1b83dc844244333baaea9e3cb255212, State: Initialized, Role: FOLLOWER
I20260809 12:59:46.654098 6232 consensus_queue.cc:260] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } }
I20260809 12:59:46.655215 6210 heartbeater.cc:503] Master 127.21.235.126:40589 was elected leader, sending a full tablet report...
I20260809 12:59:46.655815 6232 ts_tablet_manager.cc:1428] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260809 12:59:46.656352 6231 raft_consensus.cc:348] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } }
I20260809 12:59:46.656787 6231 raft_consensus.cc:374] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:59:46.656904 6231 raft_consensus.cc:729] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddd38535e34b4ffb863ee3126190ab3d, State: Initialized, Role: FOLLOWER
I20260809 12:59:46.657469 6231 consensus_queue.cc:260] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } }
I20260809 12:59:46.659394 6067 heartbeater.cc:503] Master 127.21.235.126:40589 was elected leader, sending a full tablet report...
I20260809 12:59:46.659252 6233 raft_consensus.cc:348] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } }
I20260809 12:59:46.659514 6231 ts_tablet_manager.cc:1428] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260809 12:59:46.659657 6233 raft_consensus.cc:374] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:59:46.659767 6233 raft_consensus.cc:729] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6444b55f854350a5b10bbc6203df96, State: Initialized, Role: FOLLOWER
I20260809 12:59:46.660310 6233 consensus_queue.cc:260] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } }
I20260809 12:59:46.661933 6233 ts_tablet_manager.cc:1428] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96: Time spent starting tablet: real 0.013s user 0.005s sys 0.009s
W20260809 12:59:46.720510 5921 tablet.cc:2369] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:59:46.736513 6068 tablet.cc:2369] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:59:46.775425 6211 tablet.cc:2369] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:59:46.853969 6237 raft_consensus.cc:482] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:59:46.854271 6237 raft_consensus.cc:504] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } }
I20260809 12:59:46.856200 6237 leader_election.cc:288] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ddd38535e34b4ffb863ee3126190ab3d (127.21.235.66:44063), 8a6444b55f854350a5b10bbc6203df96 (127.21.235.65:42643)
I20260809 12:59:46.862761 6225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:46.856722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:49461 (local address 127.21.235.66:44063)
0809 12:59:46.856934 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:46.856941 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:46.856966 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:59:46.857223 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:46.857231 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:46.857323 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:46.857445 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:59:46.857975 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.858824 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.860699 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.861888 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.862012 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:46.862067 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:46.862467 (+ 400us) server_negotiation.cc:299] Negotiation successful
0809 12:59:46.862564 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 12:59:46.862860 6213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:46.856437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:44063 (local address 127.21.235.67:49461)
0809 12:59:46.856851 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:46.856879 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:46.857048 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:46.857506 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:46.857510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:46.857535 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:59:46.857789 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:46.857802 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.858970 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.858976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:46.860530 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:46.860542 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.862045 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.862051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:46.862244 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:46.862283 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 12:59:46.862389 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 12:59:46.862568 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":3}
I20260809 12:59:46.863895 6228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:46.856869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:49643 (local address 127.21.235.65:42643)
0809 12:59:46.857095 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:46.857102 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:46.857640 (+ 538us) server_negotiation.cc:407] Connection header received
0809 12:59:46.857811 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:46.857816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:46.857901 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:46.858020 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:59:46.858565 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.859526 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.861687 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.862861 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.863053 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:46.863132 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:46.863609 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 12:59:46.863708 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:59:46.864074 6240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:46.856461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:42643 (local address 127.21.235.67:49643)
0809 12:59:46.857517 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:46.857534 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:46.857666 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:46.858041 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:46.858045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:46.858067 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:59:46.858358 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:46.858366 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.859700 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.859704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:46.861523 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:46.861539 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.863255 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.863260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:46.863405 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:46.863436 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 12:59:46.863564 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 12:59:46.863696 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":928,"mutex_wait_us":142,"thread_start_us":511,"threads_started":1}
I20260809 12:59:46.863935 6018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab3a694eecb4125bfd125ccbf90559d" candidate_uuid: "b1b83dc844244333baaea9e3cb255212" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd38535e34b4ffb863ee3126190ab3d" is_pre_election: true
I20260809 12:59:46.864595 6018 raft_consensus.cc:2454] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1b83dc844244333baaea9e3cb255212 in term 0.
I20260809 12:59:46.865280 6091 leader_election.cc:302] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1b83dc844244333baaea9e3cb255212, ddd38535e34b4ffb863ee3126190ab3d; no voters:
I20260809 12:59:46.865262 5871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab3a694eecb4125bfd125ccbf90559d" candidate_uuid: "b1b83dc844244333baaea9e3cb255212" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6444b55f854350a5b10bbc6203df96" is_pre_election: true
I20260809 12:59:46.865689 6237 raft_consensus.cc:2781] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:59:46.865863 6237 raft_consensus.cc:482] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:59:46.865993 6237 raft_consensus.cc:3037] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:59:46.865916 5871 raft_consensus.cc:2454] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1b83dc844244333baaea9e3cb255212 in term 0.
I20260809 12:59:46.868477 6237 raft_consensus.cc:504] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } }
I20260809 12:59:46.869053 6237 leader_election.cc:288] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [CANDIDATE]: Term 1 election: Requested vote from peers ddd38535e34b4ffb863ee3126190ab3d (127.21.235.66:44063), 8a6444b55f854350a5b10bbc6203df96 (127.21.235.65:42643)
I20260809 12:59:46.869517 5871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab3a694eecb4125bfd125ccbf90559d" candidate_uuid: "b1b83dc844244333baaea9e3cb255212" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6444b55f854350a5b10bbc6203df96"
I20260809 12:59:46.869515 6018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab3a694eecb4125bfd125ccbf90559d" candidate_uuid: "b1b83dc844244333baaea9e3cb255212" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd38535e34b4ffb863ee3126190ab3d"
I20260809 12:59:46.869704 5871 raft_consensus.cc:3037] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:59:46.869724 6018 raft_consensus.cc:3037] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:59:46.871919 5871 raft_consensus.cc:2454] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1b83dc844244333baaea9e3cb255212 in term 1.
I20260809 12:59:46.871970 6018 raft_consensus.cc:2454] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1b83dc844244333baaea9e3cb255212 in term 1.
I20260809 12:59:46.872372 6090 leader_election.cc:302] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6444b55f854350a5b10bbc6203df96, b1b83dc844244333baaea9e3cb255212; no voters:
I20260809 12:59:46.872696 6237 raft_consensus.cc:2781] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:59:46.873135 6237 raft_consensus.cc:686] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [term 1 LEADER]: Becoming Leader. State: Replica: b1b83dc844244333baaea9e3cb255212, State: Running, Role: LEADER
I20260809 12:59:46.873601 6237 consensus_queue.cc:237] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } }
I20260809 12:59:46.878361 5710 catalog_manager.cc:5495] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1b83dc844244333baaea9e3cb255212 (127.21.235.67). New cstate: current_term: 1 leader_uuid: "b1b83dc844244333baaea9e3cb255212" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } health_report { overall_health: HEALTHY } } }
I20260809 12:59:46.900794 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:46.895090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50410 (local address 127.21.235.67:38663)
0809 12:59:46.895286 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:46.895293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:46.895400 (+ 107us) server_negotiation.cc:407] Connection header received
0809 12:59:46.895634 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:46.895638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:46.895721 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:46.895870 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:59:46.896257 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.896995 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.898708 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.899105 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.899243 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:46.899742 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:59:46.900217 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:59:46.900397 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:46.900459 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 12:59:46.900537 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 12:59:46.908808 6018 raft_consensus.cc:1261] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d [term 1 FOLLOWER]: Refusing update from remote peer b1b83dc844244333baaea9e3cb255212: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:59:46.908857 5871 raft_consensus.cc:1261] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Refusing update from remote peer b1b83dc844244333baaea9e3cb255212: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:59:46.909833 6241 consensus_queue.cc:1035] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:59:46.910177 6237 consensus_queue.cc:1035] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:59:46.923002 6245 mvcc.cc:205] Tried to move back new op lower bound from 7316604464767987712 to 7316604464637931520. Current Snapshot: MvccSnapshot[applied={T|T < 7316604464767987712}]
I20260809 12:59:46.935101 5712 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:59:46.935374 5712 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:59:46.939041 5712 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d: Sending DeleteTablet for 3 replicas of tablet cab3a694eecb4125bfd125ccbf90559d
I20260809 12:59:46.940237 5994 tablet_service.cc:1514] Processing DeleteTablet for tablet cab3a694eecb4125bfd125ccbf90559d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:59:46 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:47522
I20260809 12:59:46.940233 5849 tablet_service.cc:1514] Processing DeleteTablet for tablet cab3a694eecb4125bfd125ccbf90559d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:59:46 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:54518
I20260809 12:59:46.941561 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 5778
I20260809 12:59:46.942688 6140 tablet_service.cc:1514] Processing DeleteTablet for tablet cab3a694eecb4125bfd125ccbf90559d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:59:46 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:50406
I20260809 12:59:46.950553 6257 tablet_replica.cc:330] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212: stopping tablet replica
I20260809 12:59:46.951862 6257 raft_consensus.cc:2229] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:59:46.952725 6257 raft_consensus.cc:2258] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:59:46.957288 6255 tablet_replica.cc:330] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d: stopping tablet replica
I20260809 12:59:46.957811 6255 raft_consensus.cc:2229] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:59:46.958395 6255 raft_consensus.cc:2258] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:59:46.959295 6254 tablet_replica.cc:330] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96: stopping tablet replica
I20260809 12:59:46.959738 6254 raft_consensus.cc:2229] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:59:46.960346 6254 raft_consensus.cc:2258] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:59:46.961483 6257 ts_tablet_manager.cc:1905] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:59:46.969201 6257 ts_tablet_manager.cc:1918] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:59:46.969413 6257 log.cc:1196] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/cab3a694eecb4125bfd125ccbf90559d
I20260809 12:59:46.970003 6257 ts_tablet_manager.cc:1939] T cab3a694eecb4125bfd125ccbf90559d P b1b83dc844244333baaea9e3cb255212: Deleting consensus metadata
Aug 09 12:59:46 dist-test-slave-gp6n krb5kdc[5664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 12:59:46.972239 6254 ts_tablet_manager.cc:1905] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:59:46.976197 5697 catalog_manager.cc:4841] TS b1b83dc844244333baaea9e3cb255212 (127.21.235.67:38663): tablet cab3a694eecb4125bfd125ccbf90559d (table test-table [id=94511486c6f7417592fe2fe98dc4081d]) successfully deleted
I20260809 12:59:46.976431 6255 ts_tablet_manager.cc:1905] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:59:46.978586 6254 ts_tablet_manager.cc:1918] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:59:46.978749 6254 log.cc:1196] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/cab3a694eecb4125bfd125ccbf90559d
I20260809 12:59:46.979281 6254 ts_tablet_manager.cc:1939] T cab3a694eecb4125bfd125ccbf90559d P 8a6444b55f854350a5b10bbc6203df96: Deleting consensus metadata
I20260809 12:59:46.981196 5694 catalog_manager.cc:4841] TS 8a6444b55f854350a5b10bbc6203df96 (127.21.235.65:42643): tablet cab3a694eecb4125bfd125ccbf90559d (table test-table [id=94511486c6f7417592fe2fe98dc4081d]) successfully deleted
I20260809 12:59:46.983016 6228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:46.943354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54534 (local address 127.21.235.65:42643)
0809 12:59:46.943600 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:46.943610 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:46.959637 (+ 16027us) server_negotiation.cc:407] Connection header received
0809 12:59:46.959894 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:46.959900 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:46.959994 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:46.960115 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:46.962251 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.963432 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.965411 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:46.966096 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:46.966281 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:46.972107 (+ 5826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:46.972122 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:46.972140 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:46.972181 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:46.978459 (+ 6278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:46.979699 (+ 1240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:46.979710 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:46.979716 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:46.979878 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:46.980399 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:46.980403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:46.980407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:46.980894 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:46.981876 (+ 982us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:46.982043 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 12:59:46.982648 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 12:59:46.983668 6255 ts_tablet_manager.cc:1918] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:59:46.983853 6255 log.cc:1196] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/cab3a694eecb4125bfd125ccbf90559d
I20260809 12:59:46.984254 5915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:46.984479 6255 ts_tablet_manager.cc:1939] T cab3a694eecb4125bfd125ccbf90559d P ddd38535e34b4ffb863ee3126190ab3d: Deleting consensus metadata
I20260809 12:59:46.987039 5695 catalog_manager.cc:4841] TS ddd38535e34b4ffb863ee3126190ab3d (127.21.235.66:44063): tablet cab3a694eecb4125bfd125ccbf90559d (table test-table [id=94511486c6f7417592fe2fe98dc4081d]) successfully deleted
I20260809 12:59:47.079694 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 5778
I20260809 12:59:47.101908 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 5926
Aug 09 12:59:47 dist-test-slave-gp6n krb5kdc[5664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.66@KRBTEST.COM
I20260809 12:59:47.120790 6225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:47.102905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47532 (local address 127.21.235.66:44063)
0809 12:59:47.103088 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:47.103093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:47.103162 (+ 69us) server_negotiation.cc:407] Connection header received
0809 12:59:47.103358 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:47.103362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:47.103426 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:47.103512 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:47.105479 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:47.106264 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:47.108438 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:47.108845 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:47.108940 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:47.114574 (+ 5634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:47.114590 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:47.114605 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:47.114648 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:47.117546 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:47.118274 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:47.118282 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:47.118285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:47.118674 (+ 389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:47.119081 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:47.119086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:47.119090 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:47.119580 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:47.119891 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:47.120204 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 12:59:47.120443 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 12:59:47.121438 6062 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:47.215672 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 5926
I20260809 12:59:47.234594 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 6072
Aug 09 12:59:47 dist-test-slave-gp6n krb5kdc[5664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.67@KRBTEST.COM
I20260809 12:59:47.253394 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:47.235571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50420 (local address 127.21.235.67:38663)
0809 12:59:47.235792 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:47.235812 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:47.235835 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:59:47.235909 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:47.235916 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:47.236017 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:47.236132 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:47.237462 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:47.238841 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:47.240147 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:47.240916 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:47.241059 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:47.245763 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:47.245777 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:47.245796 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:47.245858 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:47.250248 (+ 4390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:47.251167 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:47.251177 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:47.251181 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:47.251364 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:47.251718 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:47.251724 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:47.251728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:47.252195 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:47.252527 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:47.252725 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 12:59:47.252958 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 12:59:47.254194 6205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:47.377378 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 6072
I20260809 12:59:47.401827 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 5677
I20260809 12:59:47.403235 5744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:47.516299 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 5677
I20260809 12:59:47.540966 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:40589
--webserver_interface=127.21.235.126
--webserver_port=43615
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:40589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:59:47.883244 6262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:47.883594 6262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:47.883749 6262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:47.891052 6262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:59:47.891196 6262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:47.891299 6262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:47.891402 6262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:59:47.891484 6262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:59:47.901091 6262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:40589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:40589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=43615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:47.902448 6262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:47.903798 6262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:47.929224 6271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:47.929276 6269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:47.929306 6268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:47.929953 6262 server_base.cc:1034] running on GCE node
I20260809 12:59:47.931010 6262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:47.932592 6262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:47.933799 6262 hybrid_clock.cc:648] HybridClock initialized: now 1786280387933757 us; error 31 us; skew 500 ppm
I20260809 12:59:47.934242 6262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:47 dist-test-slave-gp6n krb5kdc[5664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280387, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:47.937834 6262 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:59:47.939670 6262 webserver.cc:460] Webserver started at http://127.21.235.126:43615/ using document root <none> and password file <none>
I20260809 12:59:47.940435 6262 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:47.940608 6262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:47.946798 6262 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 12:59:47.949735 6278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:47.951143 6262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:59:47.951334 6262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d"
format_stamp: "Formatted at 2026-08-09 12:59:44 on dist-test-slave-gp6n"
server_key: "01f6e554a73bfdbd04ccc1245b17a941"
server_key_iv: "fb8bdf9c3910f4eafe0d61019e067e6b"
server_key_version: "encryptionkey@0"
I20260809 12:59:47.952641 6262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:47.966038 6262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:47.973016 6262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:47.973311 6262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:47.992775 6262 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:40589
I20260809 12:59:47.992810 6338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:40589 every 8 connection(s)
I20260809 12:59:47.994470 6262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:59:47.996218 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 6262
I20260809 12:59:47.997243 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:42643
--local_ip_for_outbound_sockets=127.21.235.65
--tserver_master_addrs=127.21.235.126:40589
--webserver_port=37913
--webserver_interface=127.21.235.65
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:59:48.004829 6339 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:59:48.009660 6339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d: Bootstrap starting.
I20260809 12:59:48.029307 6339 log.cc:824] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:48.041755 6339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:59:48.042315 6339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d: Bootstrap complete.
I20260809 12:59:48.051707 6339 raft_consensus.cc:348] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40589 } }
I20260809 12:59:48.052533 6339 raft_consensus.cc:729] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87bf791a9d742a9b1d0bd8e4bb3134d, State: Initialized, Role: FOLLOWER
I20260809 12:59:48.053201 6339 consensus_queue.cc:260] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40589 } }
I20260809 12:59:48.053469 6339 raft_consensus.cc:388] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:59:48.053622 6339 raft_consensus.cc:482] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:59:48.053812 6339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 1 FOLLOWER]: Advancing to term 2
I20260809 12:59:48.058532 6339 raft_consensus.cc:504] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40589 } }
I20260809 12:59:48.059108 6339 leader_election.cc:302] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c87bf791a9d742a9b1d0bd8e4bb3134d; no voters:
I20260809 12:59:48.059831 6339 leader_election.cc:288] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 12:59:48.059985 6343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 2 FOLLOWER]: Leader election won for term 2
I20260809 12:59:48.062474 6339 sys_catalog.cc:564] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:59:48.067155 6343 raft_consensus.cc:686] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 2 LEADER]: Becoming Leader. State: Replica: c87bf791a9d742a9b1d0bd8e4bb3134d, State: Running, Role: LEADER
I20260809 12:59:48.067759 6343 consensus_queue.cc:237] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40589 } }
I20260809 12:59:48.073127 6344 sys_catalog.cc:455] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40589 } } }
I20260809 12:59:48.073652 6344 sys_catalog.cc:458] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [sys.catalog]: This master's current role is: LEADER
I20260809 12:59:48.073166 6345 sys_catalog.cc:455] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c87bf791a9d742a9b1d0bd8e4bb3134d. Latest consensus state: current_term: 2 leader_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40589 } } }
I20260809 12:59:48.078895 6345 sys_catalog.cc:458] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [sys.catalog]: This master's current role is: LEADER
I20260809 12:59:48.087862 6358 catalog_manager.cc:1259] Loaded cluster ID: 5f72890e10f44d78a008594817f0213d
I20260809 12:59:48.088004 6358 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d: loading cluster ID for follower catalog manager: success
W20260809 12:59:48.089555 6358 catalog_manager.cc:1576] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260809 12:59:48.089720 6358 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260809 12:59:48.089854 6359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:59:48.094529 6359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:59:48.095701 6359 catalog_manager.cc:1259] Loaded cluster ID: 5f72890e10f44d78a008594817f0213d
I20260809 12:59:48.095798 6359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:59:48.098454 6359 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 12:59:48.115553 6359 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 12:59:48.125180 6359 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:59:48.126892 6359 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d: Loaded TSK: 0
I20260809 12:59:48.128407 6359 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 12:59:48.364542 6341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:48.364872 6341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:48.365033 6341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:48.372859 6341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:48.373040 6341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:48.373225 6341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:59:48.383543 6341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:42643
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=37913
--tserver_master_addrs=127.21.235.126:40589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:48.384964 6341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:48.386507 6341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:48.401592 6367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:48.401836 6369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:48.402473 6366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:48.402856 6341 server_base.cc:1034] running on GCE node
I20260809 12:59:48.403594 6341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:48.404707 6341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:48.405927 6341 hybrid_clock.cc:648] HybridClock initialized: now 1786280388405881 us; error 44 us; skew 500 ppm
I20260809 12:59:48.406265 6341 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:48 dist-test-slave-gp6n krb5kdc[5664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280388, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:48.409729 6341 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 12:59:48.411419 6341 webserver.cc:460] Webserver started at http://127.21.235.65:37913/ using document root <none> and password file <none>
I20260809 12:59:48.412189 6341 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:48.412417 6341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:48.418057 6341 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 12:59:48.421267 6376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:48.422824 6341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:59:48.423002 6341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "8a6444b55f854350a5b10bbc6203df96"
format_stamp: "Formatted at 2026-08-09 12:59:45 on dist-test-slave-gp6n"
server_key: "02452535ac568896337ab293a8fd4862"
server_key_iv: "c10546a5f265bfc24354472c84da4726"
server_key_version: "encryptionkey@0"
I20260809 12:59:48.424041 6341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:48.444370 6341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:48.448782 6341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:48.449069 6341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:48.450081 6341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:59:48.451536 6341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:59:48.451660 6341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:48.451800 6341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:59:48.451892 6341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:48.486905 6341 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:42643
I20260809 12:59:48.486953 6505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:42643 every 8 connection(s)
I20260809 12:59:48.488399 6341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:59:48.497617 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 6341
I20260809 12:59:48.498317 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:44063
--local_ip_for_outbound_sockets=127.21.235.66
--tserver_master_addrs=127.21.235.126:40589
--webserver_port=46425
--webserver_interface=127.21.235.66
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:59:48 dist-test-slave-gp6n krb5kdc[5664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280388, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:59:48.514156 6510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:48.491699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:32937)
0809 12:59:48.492705 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:48.492751 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:48.494681 (+ 1930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:48.496157 (+ 1476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:48.496171 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:48.496835 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:48.497436 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:48.497455 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:48.499494 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:48.499498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:48.500861 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:48.500874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:48.502004 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:48.502013 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:48.502313 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:48.503126 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:48.503156 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:48.505501 (+ 2345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:48.509388 (+ 3887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:48.509397 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:48.509413 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:48.509855 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:48.510339 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:48.510345 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:48.510349 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:48.510621 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:48.511425 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:48.511433 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:48.511820 (+ 387us) client_negotiation.cc:769] Sending connection context
0809 12:59:48.512113 (+ 293us) client_negotiation.cc:240] Negotiation successful
0809 12:59:48.512747 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":547,"threads_started":1}
I20260809 12:59:48.514158 6508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:48.491656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:32937 (local address 127.21.235.126:40589)
0809 12:59:48.492377 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:48.492396 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:48.492908 (+ 512us) server_negotiation.cc:407] Connection header received
0809 12:59:48.495245 (+ 2337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:48.495259 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:48.495600 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:48.495999 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:48.497676 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:48.499296 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:48.501016 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:48.501709 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:48.501966 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:48.505811 (+ 3845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:48.505819 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:48.505841 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:48.505880 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:48.509190 (+ 3310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:48.510022 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:48.510031 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:48.510037 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:48.510198 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:48.510747 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:48.510754 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:48.510758 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:48.511257 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:48.511471 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:48.512359 (+ 888us) server_negotiation.cc:299] Negotiation successful
0809 12:59:48.512802 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":284,"threads_started":1}
I20260809 12:59:48.517607 6506 heartbeater.cc:347] Connected to a master server at 127.21.235.126:40589
I20260809 12:59:48.517961 6506 heartbeater.cc:464] Registering TS with master...
I20260809 12:59:48.518859 6506 heartbeater.cc:511] Master 127.21.235.126:40589 requested a full tablet report, sending...
I20260809 12:59:48.521590 6299 ts_manager.cc:194] Registered new tserver with Master: 8a6444b55f854350a5b10bbc6203df96 (127.21.235.65:42643)
I20260809 12:59:48.524029 6299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:32937
W20260809 12:59:48.837143 6511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:48.837419 6511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:48.837512 6511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:48.844623 6511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:48.844725 6511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:48.844836 6511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:59:48.854099 6511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:44063
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=46425
--tserver_master_addrs=127.21.235.126:40589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:48.855497 6511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:48.856850 6511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:48.870102 6516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:48.870337 6517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:48.870342 6519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:48.871495 6511 server_base.cc:1034] running on GCE node
I20260809 12:59:48.872176 6511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:48.873070 6511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:48.874301 6511 hybrid_clock.cc:648] HybridClock initialized: now 1786280388874251 us; error 65 us; skew 500 ppm
I20260809 12:59:48.874650 6511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:48 dist-test-slave-gp6n krb5kdc[5664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280388, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:48.877775 6511 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 12:59:48.879293 6511 webserver.cc:460] Webserver started at http://127.21.235.66:46425/ using document root <none> and password file <none>
I20260809 12:59:48.879992 6511 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:48.880155 6511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:48.885476 6511 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 12:59:48.888279 6528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:48.889669 6511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:59:48.889849 6511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "ddd38535e34b4ffb863ee3126190ab3d"
format_stamp: "Formatted at 2026-08-09 12:59:45 on dist-test-slave-gp6n"
server_key: "f085f1cc929d0e379918826c255c36a8"
server_key_iv: "3ae04146b7d434443e7d91dee4ffab22"
server_key_version: "encryptionkey@0"
I20260809 12:59:48.890914 6511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:48.919883 6511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:48.923868 6511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:48.924170 6511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:48.925176 6511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:59:48.926540 6511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:59:48.926672 6511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:48.926805 6511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:59:48.926896 6511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:48.961789 6511 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:44063
I20260809 12:59:48.961841 6650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:44063 every 8 connection(s)
I20260809 12:59:48.963433 6511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:59:48.966549 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 6511
I20260809 12:59:48.967396 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:38663
--local_ip_for_outbound_sockets=127.21.235.67
--tserver_master_addrs=127.21.235.126:40589
--webserver_port=40971
--webserver_interface=127.21.235.67
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:59:48 dist-test-slave-gp6n krb5kdc[5664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280388, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:59:48.985579 6508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:48.966153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:48969 (local address 127.21.235.126:40589)
0809 12:59:48.966335 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:48.966340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:48.968000 (+ 1660us) server_negotiation.cc:407] Connection header received
0809 12:59:48.969917 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:48.969923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:48.970028 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:48.970144 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:48.971740 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:48.972448 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:48.974033 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:48.974558 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:48.974710 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:48.978493 (+ 3783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:48.978502 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:48.978510 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:48.978556 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:48.982141 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:48.982919 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:48.982927 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:48.982933 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:48.983104 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:48.983517 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:48.983524 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:48.983531 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:48.983934 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:48.984103 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:48.984833 (+ 730us) server_negotiation.cc:299] Negotiation successful
0809 12:59:48.985122 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 12:59:48.986289 6653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:48.966873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:48969)
0809 12:59:48.967824 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:48.967870 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:48.969638 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:48.970393 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:48.970446 (+ 53us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:48.970976 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:48.971558 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:48.971576 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:48.972607 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:48.972619 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:48.973859 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:48.973871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:48.974926 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:48.974931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:48.975129 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:48.975815 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:48.975838 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:48.978245 (+ 2407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:48.982289 (+ 4044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:48.982296 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:48.982308 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:48.982800 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:48.983215 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:48.983218 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:48.983220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:48.983399 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:48.984088 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:48.984094 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:48.984398 (+ 304us) client_negotiation.cc:769] Sending connection context
0809 12:59:48.984658 (+ 260us) client_negotiation.cc:240] Negotiation successful
0809 12:59:48.985050 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":321,"threads_started":1}
I20260809 12:59:48.987866 6651 heartbeater.cc:347] Connected to a master server at 127.21.235.126:40589
I20260809 12:59:48.988179 6651 heartbeater.cc:464] Registering TS with master...
I20260809 12:59:48.988916 6651 heartbeater.cc:511] Master 127.21.235.126:40589 requested a full tablet report, sending...
I20260809 12:59:48.990620 6299 ts_manager.cc:194] Registered new tserver with Master: ddd38535e34b4ffb863ee3126190ab3d (127.21.235.66:44063)
I20260809 12:59:48.991452 6299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:48969
W20260809 12:59:49.311832 6654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:49.312098 6654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:49.312283 6654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:49.319545 6654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:49.319666 6654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:49.319852 6654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:59:49.329774 6654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:38663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=40971
--tserver_master_addrs=127.21.235.126:40589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:49.331090 6654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:49.332415 6654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:49.345610 6660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:49.345700 6661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:49.345655 6663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:49.351614 6654 server_base.cc:1034] running on GCE node
I20260809 12:59:49.352223 6654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:49.353164 6654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:49.354473 6654 hybrid_clock.cc:648] HybridClock initialized: now 1786280389354374 us; error 48 us; skew 500 ppm
I20260809 12:59:49.354803 6654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:49 dist-test-slave-gp6n krb5kdc[5664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280389, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:49.358392 6654 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 12:59:49.360057 6654 webserver.cc:460] Webserver started at http://127.21.235.67:40971/ using document root <none> and password file <none>
I20260809 12:59:49.360790 6654 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:49.360956 6654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:49.366978 6654 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 12:59:49.374631 6670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:49.376300 6654 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.004s sys 0.004s
I20260809 12:59:49.376438 6654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "b1b83dc844244333baaea9e3cb255212"
format_stamp: "Formatted at 2026-08-09 12:59:46 on dist-test-slave-gp6n"
server_key: "6eab2ef5aa3b3eeaf9629545c6605e66"
server_key_iv: "61b3a1291d8565ee5d7bcc75dc8a3bce"
server_key_version: "encryptionkey@0"
I20260809 12:59:49.377434 6654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:49.414589 6654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:49.419071 6654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:49.419306 6654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:49.420357 6654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:59:49.421720 6654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:59:49.421808 6654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:49.421881 6654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:59:49.421947 6654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:49.455075 6654 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:38663
I20260809 12:59:49.455150 6796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:38663 every 8 connection(s)
I20260809 12:59:49.456738 6654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:59:49.457813 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 6654
Aug 09 12:59:49 dist-test-slave-gp6n krb5kdc[5664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280389, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 12:59:49.477620 6508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.458934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57182 (local address 127.21.235.126:40589)
0809 12:59:49.459146 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:49.459153 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:49.461884 (+ 2731us) server_negotiation.cc:407] Connection header received
0809 12:59:49.462038 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:49.462044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:49.462131 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:49.462253 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:49.464094 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.465205 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.467228 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.467865 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.468013 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:49.471147 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:49.471154 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:49.471159 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:49.471193 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:49.474500 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:49.475432 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:49.475442 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:49.475446 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:49.475604 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:49.476123 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:49.476133 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:49.476141 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:49.476423 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:49.476738 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:49.477076 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 12:59:49.477292 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 12:59:49.480679 6802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.459705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:34347 (local address 127.21.235.126:40589)
0809 12:59:49.460217 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:49.460222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:49.461234 (+ 1012us) server_negotiation.cc:407] Connection header received
0809 12:59:49.463503 (+ 2269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:49.463514 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:49.463581 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:49.463696 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:49.465681 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.466804 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.469030 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.469807 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.469949 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:49.474102 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:49.474107 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:49.474110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:49.474149 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:49.477131 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:49.478008 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:49.478012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:49.478015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:49.478148 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:49.478696 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:49.478701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:49.478705 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:49.479052 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:49.479208 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:49.480087 (+ 879us) server_negotiation.cc:299] Negotiation successful
0809 12:59:49.480308 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":271,"threads_started":1}
I20260809 12:59:49.481603 6803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.460228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:34347)
0809 12:59:49.461052 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:49.461104 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:49.463147 (+ 2043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:49.463974 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:49.463987 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:49.464703 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:49.465488 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:49.465517 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.466965 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.466969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:49.468894 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:49.468906 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.470031 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.470036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:49.470226 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:49.470888 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:49.470909 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:49.473860 (+ 2951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:49.477273 (+ 3413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:49.477281 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:49.477297 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:49.477889 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:49.478268 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:49.478275 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:49.478280 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:49.478579 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:49.479201 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:49.479207 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:49.479670 (+ 463us) client_negotiation.cc:769] Sending connection context
0809 12:59:49.479968 (+ 298us) client_negotiation.cc:240] Negotiation successful
0809 12:59:49.480491 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":339,"threads_started":1}
I20260809 12:59:49.483084 6797 heartbeater.cc:347] Connected to a master server at 127.21.235.126:40589
I20260809 12:59:49.483340 6797 heartbeater.cc:464] Registering TS with master...
I20260809 12:59:49.484011 6797 heartbeater.cc:511] Master 127.21.235.126:40589 requested a full tablet report, sending...
I20260809 12:59:49.485296 6299 ts_manager.cc:194] Registered new tserver with Master: b1b83dc844244333baaea9e3cb255212 (127.21.235.67:38663)
I20260809 12:59:49.486088 6299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:34347
I20260809 12:59:49.492476 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:59:49.499342 6802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.493337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57184 (local address 127.21.235.126:40589)
0809 12:59:49.493528 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:49.493536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:49.494017 (+ 481us) server_negotiation.cc:407] Connection header received
0809 12:59:49.494243 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:49.494248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:49.494392 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:49.494584 (+ 192us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:59:49.495040 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.495719 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.497215 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.497636 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.497771 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:49.498263 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:59:49.498783 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:59:49.498960 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:49.499013 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 12:59:49.499089 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 12:59:49.500162 6299 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:57184:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:59:49.507473 6299 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:59:49.527644 6506 heartbeater.cc:503] Master 127.21.235.126:40589 was elected leader, sending a full tablet report...
I20260809 12:59:49.553833 6809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.518345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:38663 (local address 127.0.0.1:50422)
0809 12:59:49.539431 (+ 21086us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:49.539485 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:49.539654 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:49.546251 (+ 6597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:49.546255 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:49.546286 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:59:49.547202 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:49.547218 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.549299 (+ 2081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.549306 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:49.551206 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:49.551222 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.552766 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.552774 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:49.552993 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:49.553055 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 12:59:49.553186 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 12:59:49.553391 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20797,"mutex_wait_us":84,"thread_start_us":20672,"threads_started":1}
I20260809 12:59:49.553823 6810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.518658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50422 (local address 127.21.235.67:38663)
0809 12:59:49.545287 (+ 26629us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:49.545299 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:49.545334 (+ 35us) server_negotiation.cc:407] Connection header received
0809 12:59:49.545437 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:49.545444 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:49.545700 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:49.545899 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:59:49.547456 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.549083 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.551376 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.552551 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.552688 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:49.552927 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:49.553279 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 12:59:49.553456 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26435,"thread_start_us":26257,"threads_started":1}
I20260809 12:59:49.554733 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.518750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:42643 (local address 127.0.0.1:54542)
0809 12:59:49.544408 (+ 25658us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:49.544432 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:49.544578 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:49.546365 (+ 1787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:49.546370 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:49.546393 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:59:49.547349 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:49.547361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.549114 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.549121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:49.551882 (+ 2761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:49.551898 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.553848 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.553857 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:49.554063 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:49.554092 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 12:59:49.554171 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 12:59:49.554291 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25495,"mutex_wait_us":192,"spinlock_wait_cycles":4480,"thread_start_us":26112,"threads_started":1}
I20260809 12:59:49.554777 6812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.518779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54542 (local address 127.21.235.65:42643)
0809 12:59:49.545607 (+ 26828us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:49.545615 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:49.545639 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:59:49.545733 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:49.545742 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:49.545938 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:49.546134 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:59:49.547542 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.548985 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.552083 (+ 3098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.553299 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.553422 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:49.553665 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:49.554267 (+ 602us) server_negotiation.cc:299] Negotiation successful
0809 12:59:49.554467 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26657,"thread_start_us":26575,"threads_started":1}
I20260809 12:59:49.555022 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.519528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47534 (local address 127.21.235.66:44063)
0809 12:59:49.543665 (+ 24137us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:49.543678 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:49.545292 (+ 1614us) server_negotiation.cc:407] Connection header received
0809 12:59:49.545411 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:49.545420 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:49.545707 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:49.545901 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:59:49.547365 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.549083 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.552064 (+ 2981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.553445 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.553605 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:49.553786 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:49.554523 (+ 737us) server_negotiation.cc:299] Negotiation successful
0809 12:59:49.554690 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23919,"thread_start_us":24168,"threads_started":1}
I20260809 12:59:49.558370 6724 tablet_service.cc:1467] Processing CreateTablet for tablet ea02aabda4b6465f954455a3e72319c0 (DEFAULT_TABLE table=test-table [id=d571cc93c1d746eaac08c9a8c825986e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:59:49.558791 6432 tablet_service.cc:1467] Processing CreateTablet for tablet ea02aabda4b6465f954455a3e72319c0 (DEFAULT_TABLE table=test-table [id=d571cc93c1d746eaac08c9a8c825986e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:59:49.559732 6724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea02aabda4b6465f954455a3e72319c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:59:49.560001 6432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea02aabda4b6465f954455a3e72319c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:59:49.562487 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.518957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:44063 (local address 127.0.0.1:47534)
0809 12:59:49.545117 (+ 26160us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:49.545139 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:49.545255 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:49.546177 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:49.546184 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:49.546208 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:59:49.547129 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:49.547150 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.549213 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.549219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:49.551882 (+ 2663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:49.551898 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.553965 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.553975 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:49.554148 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:49.554189 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 12:59:49.554272 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 12:59:49.555031 (+ 759us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26043,"mutex_wait_us":80,"thread_start_us":26116,"threads_started":1}
I20260809 12:59:49.565587 6578 tablet_service.cc:1467] Processing CreateTablet for tablet ea02aabda4b6465f954455a3e72319c0 (DEFAULT_TABLE table=test-table [id=d571cc93c1d746eaac08c9a8c825986e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:59:49.566809 6578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea02aabda4b6465f954455a3e72319c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:59:49.573472 6815 tablet_bootstrap.cc:492] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96: Bootstrap starting.
I20260809 12:59:49.576982 6816 tablet_bootstrap.cc:492] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212: Bootstrap starting.
I20260809 12:59:49.577291 6815 tablet_bootstrap.cc:654] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96: Neither blocks nor log segments found. Creating new log.
I20260809 12:59:49.578637 6815 log.cc:824] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:49.580329 6816 tablet_bootstrap.cc:654] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212: Neither blocks nor log segments found. Creating new log.
I20260809 12:59:49.581107 6819 tablet_bootstrap.cc:492] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d: Bootstrap starting.
I20260809 12:59:49.581784 6816 log.cc:824] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:49.581930 6815 tablet_bootstrap.cc:492] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96: No bootstrap required, opened a new log
I20260809 12:59:49.582214 6815 ts_tablet_manager.cc:1397] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 12:59:49.584367 6816 tablet_bootstrap.cc:492] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212: No bootstrap required, opened a new log
I20260809 12:59:49.584640 6816 ts_tablet_manager.cc:1397] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.002s
I20260809 12:59:49.584895 6819 tablet_bootstrap.cc:654] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d: Neither blocks nor log segments found. Creating new log.
I20260809 12:59:49.586107 6819 log.cc:824] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:49.589157 6819 tablet_bootstrap.cc:492] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d: No bootstrap required, opened a new log
I20260809 12:59:49.589434 6819 ts_tablet_manager.cc:1397] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 12:59:49.592331 6816 raft_consensus.cc:348] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } }
I20260809 12:59:49.592689 6816 raft_consensus.cc:374] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:59:49.592788 6816 raft_consensus.cc:729] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1b83dc844244333baaea9e3cb255212, State: Initialized, Role: FOLLOWER
I20260809 12:59:49.592617 6815 raft_consensus.cc:348] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } }
I20260809 12:59:49.593048 6815 raft_consensus.cc:374] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:59:49.593184 6815 raft_consensus.cc:729] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6444b55f854350a5b10bbc6203df96, State: Initialized, Role: FOLLOWER
I20260809 12:59:49.593447 6816 consensus_queue.cc:260] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } }
I20260809 12:59:49.593806 6815 consensus_queue.cc:260] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } }
I20260809 12:59:49.594625 6797 heartbeater.cc:503] Master 127.21.235.126:40589 was elected leader, sending a full tablet report...
I20260809 12:59:49.595374 6816 ts_tablet_manager.cc:1428] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212: Time spent starting tablet: real 0.011s user 0.009s sys 0.003s
I20260809 12:59:49.600955 6815 ts_tablet_manager.cc:1428] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96: Time spent starting tablet: real 0.019s user 0.012s sys 0.006s
I20260809 12:59:49.601693 6819 raft_consensus.cc:348] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } }
I20260809 12:59:49.602080 6819 raft_consensus.cc:374] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:59:49.602187 6819 raft_consensus.cc:729] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddd38535e34b4ffb863ee3126190ab3d, State: Initialized, Role: FOLLOWER
I20260809 12:59:49.602797 6819 consensus_queue.cc:260] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } }
I20260809 12:59:49.603950 6651 heartbeater.cc:503] Master 127.21.235.126:40589 was elected leader, sending a full tablet report...
I20260809 12:59:49.604408 6819 ts_tablet_manager.cc:1428] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
I20260809 12:59:49.656399 6823 raft_consensus.cc:482] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:59:49.656787 6823 raft_consensus.cc:504] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } }
I20260809 12:59:49.658783 6823 leader_election.cc:288] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1b83dc844244333baaea9e3cb255212 (127.21.235.67:38663), ddd38535e34b4ffb863ee3126190ab3d (127.21.235.66:44063)
I20260809 12:59:49.665823 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.659389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:53959 (local address 127.21.235.66:44063)
0809 12:59:49.659651 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:49.659657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:49.660132 (+ 475us) server_negotiation.cc:407] Connection header received
0809 12:59:49.660323 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:49.660327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:49.660387 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:49.660489 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:59:49.661063 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.661773 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.664115 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.664872 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.665060 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:49.665140 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:49.665434 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 12:59:49.665560 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260809 12:59:49.665985 6826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.659139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:38663 (local address 127.21.235.65:40371)
0809 12:59:49.660105 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:49.660126 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:49.660264 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:49.660597 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:49.660602 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:49.660642 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:59:49.661055 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:49.661064 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.662084 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.662088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:49.663807 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:49.663840 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.664998 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.665004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:49.665177 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:49.665205 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 12:59:49.665312 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 12:59:49.665436 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"mutex_wait_us":2,"thread_start_us":676,"threads_started":1}
I20260809 12:59:49.665717 6810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.659136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:40371 (local address 127.21.235.67:38663)
0809 12:59:49.659358 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:49.659367 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:49.660203 (+ 836us) server_negotiation.cc:407] Connection header received
0809 12:59:49.660412 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:49.660417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:49.660477 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:49.660576 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:59:49.661215 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.661913 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.664021 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.664780 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.664982 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:49.665057 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:49.665338 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 12:59:49.665462 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 12:59:49.666124 6825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.659113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:44063 (local address 127.21.235.65:53959)
0809 12:59:49.659986 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:49.660008 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:49.660161 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:49.660504 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:49.660509 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:49.660532 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:59:49.660914 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:49.660924 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.661899 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.661907 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:49.663885 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:49.663897 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.665102 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.665108 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:49.665279 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:49.665301 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 12:59:49.665406 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 12:59:49.665534 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"mutex_wait_us":99,"thread_start_us":461,"threads_started":1}
I20260809 12:59:49.667169 6599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea02aabda4b6465f954455a3e72319c0" candidate_uuid: "8a6444b55f854350a5b10bbc6203df96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd38535e34b4ffb863ee3126190ab3d" is_pre_election: true
I20260809 12:59:49.667212 6747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea02aabda4b6465f954455a3e72319c0" candidate_uuid: "8a6444b55f854350a5b10bbc6203df96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1b83dc844244333baaea9e3cb255212" is_pre_election: true
I20260809 12:59:49.667757 6747 raft_consensus.cc:2454] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6444b55f854350a5b10bbc6203df96 in term 0.
I20260809 12:59:49.667747 6599 raft_consensus.cc:2454] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6444b55f854350a5b10bbc6203df96 in term 0.
I20260809 12:59:49.668403 6379 leader_election.cc:302] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6444b55f854350a5b10bbc6203df96, ddd38535e34b4ffb863ee3126190ab3d; no voters:
I20260809 12:59:49.668823 6823 raft_consensus.cc:2781] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:59:49.668991 6823 raft_consensus.cc:482] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:59:49.669106 6823 raft_consensus.cc:3037] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:59:49.671339 6823 raft_consensus.cc:504] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } }
I20260809 12:59:49.671985 6823 leader_election.cc:288] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 1 election: Requested vote from peers b1b83dc844244333baaea9e3cb255212 (127.21.235.67:38663), ddd38535e34b4ffb863ee3126190ab3d (127.21.235.66:44063)
I20260809 12:59:49.672399 6747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea02aabda4b6465f954455a3e72319c0" candidate_uuid: "8a6444b55f854350a5b10bbc6203df96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1b83dc844244333baaea9e3cb255212"
I20260809 12:59:49.672626 6747 raft_consensus.cc:3037] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:59:49.672674 6599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea02aabda4b6465f954455a3e72319c0" candidate_uuid: "8a6444b55f854350a5b10bbc6203df96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd38535e34b4ffb863ee3126190ab3d"
I20260809 12:59:49.672911 6599 raft_consensus.cc:3037] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:59:49.675009 6747 raft_consensus.cc:2454] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6444b55f854350a5b10bbc6203df96 in term 1.
I20260809 12:59:49.675401 6380 leader_election.cc:302] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6444b55f854350a5b10bbc6203df96, b1b83dc844244333baaea9e3cb255212; no voters:
I20260809 12:59:49.675745 6823 raft_consensus.cc:2781] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:59:49.676082 6599 raft_consensus.cc:2454] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6444b55f854350a5b10bbc6203df96 in term 1.
I20260809 12:59:49.676249 6823 raft_consensus.cc:686] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6444b55f854350a5b10bbc6203df96, State: Running, Role: LEADER
I20260809 12:59:49.676801 6823 consensus_queue.cc:237] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } }
I20260809 12:59:49.681674 6299 catalog_manager.cc:5495] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6444b55f854350a5b10bbc6203df96 (127.21.235.65). New cstate: current_term: 1 leader_uuid: "8a6444b55f854350a5b10bbc6203df96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } health_report { overall_health: UNKNOWN } } }
W20260809 12:59:49.708431 6800 tablet.cc:2369] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:59:49.715493 6652 tablet.cc:2369] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:59:49.724507 6812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.719493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54548 (local address 127.21.235.65:42643)
0809 12:59:49.719692 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:49.719697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:49.719715 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:59:49.719800 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:49.719804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:49.719886 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:49.720076 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:59:49.720396 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.721111 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.722584 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.723001 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.723092 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:49.723542 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:59:49.724043 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:59:49.724152 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:49.724248 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 12:59:49.724313 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:59:49.733031 6747 raft_consensus.cc:1261] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212 [term 1 FOLLOWER]: Refusing update from remote peer 8a6444b55f854350a5b10bbc6203df96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:59:49.733031 6599 raft_consensus.cc:1261] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d [term 1 FOLLOWER]: Refusing update from remote peer 8a6444b55f854350a5b10bbc6203df96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:59:49.734148 6823 consensus_queue.cc:1035] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:59:49.734524 6828 consensus_queue.cc:1035] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 12:59:49.740845 6507 tablet.cc:2369] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:59:49.747622 6832 mvcc.cc:205] Tried to move back new op lower bound from 7316604476335403008 to 7316604476120072192. Current Snapshot: MvccSnapshot[applied={T|T < 7316604476335403008}]
I20260809 12:59:49.761278 6297 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:57184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:59:49.761613 6297 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:57184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:59:49.765686 6297 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d: Sending DeleteTablet for 3 replicas of tablet ea02aabda4b6465f954455a3e72319c0
I20260809 12:59:49.766831 6432 tablet_service.cc:1514] Processing DeleteTablet for tablet ea02aabda4b6465f954455a3e72319c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:59:49 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:54542
I20260809 12:59:49.766948 6724 tablet_service.cc:1514] Processing DeleteTablet for tablet ea02aabda4b6465f954455a3e72319c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:59:49 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:50422
I20260809 12:59:49.766919 6578 tablet_service.cc:1514] Processing DeleteTablet for tablet ea02aabda4b6465f954455a3e72319c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:59:49 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:47534
I20260809 12:59:49.770771 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 6341
I20260809 12:59:49.778504 6843 tablet_replica.cc:330] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212: stopping tablet replica
I20260809 12:59:49.778961 6843 raft_consensus.cc:2229] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:59:49.779477 6843 raft_consensus.cc:2258] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:59:49.782235 6843 ts_tablet_manager.cc:1905] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:59:49.790100 6843 ts_tablet_manager.cc:1918] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:59:49.790297 6843 log.cc:1196] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/ea02aabda4b6465f954455a3e72319c0
I20260809 12:59:49.791018 6843 ts_tablet_manager.cc:1939] T ea02aabda4b6465f954455a3e72319c0 P b1b83dc844244333baaea9e3cb255212: Deleting consensus metadata
I20260809 12:59:49.793094 6281 catalog_manager.cc:4841] TS b1b83dc844244333baaea9e3cb255212 (127.21.235.67:38663): tablet ea02aabda4b6465f954455a3e72319c0 (table test-table [id=d571cc93c1d746eaac08c9a8c825986e]) successfully deleted
I20260809 12:59:49.794092 6844 tablet_replica.cc:330] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d: stopping tablet replica
I20260809 12:59:49.794600 6844 raft_consensus.cc:2229] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:59:49.795173 6844 raft_consensus.cc:2258] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:59:49.795224 6812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.771769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54560 (local address 127.21.235.65:42643)
0809 12:59:49.771983 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:49.771992 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:49.772532 (+ 540us) server_negotiation.cc:407] Connection header received
0809 12:59:49.772620 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:49.772628 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:49.772732 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:49.772839 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:49.775459 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.777020 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.779668 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.780335 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.780473 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:49.784859 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:49.784874 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:49.784892 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:49.784946 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:49.789258 (+ 4312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:49.791396 (+ 2138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:49.791405 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:49.791408 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:49.791609 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:49.792161 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:49.792168 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:49.792171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:49.792641 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:49.792943 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:49.794578 (+ 1635us) server_negotiation.cc:299] Negotiation successful
0809 12:59:49.794857 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:59:49.795994 6501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:49.797850 6844 ts_tablet_manager.cc:1905] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:59:49.804507 6844 ts_tablet_manager.cc:1918] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:59:49.804692 6844 log.cc:1196] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/ea02aabda4b6465f954455a3e72319c0
I20260809 12:59:49.805207 6844 ts_tablet_manager.cc:1939] T ea02aabda4b6465f954455a3e72319c0 P ddd38535e34b4ffb863ee3126190ab3d: Deleting consensus metadata
I20260809 12:59:49.806856 6280 catalog_manager.cc:4841] TS ddd38535e34b4ffb863ee3126190ab3d (127.21.235.66:44063): tablet ea02aabda4b6465f954455a3e72319c0 (table test-table [id=d571cc93c1d746eaac08c9a8c825986e]) successfully deleted
I20260809 12:59:49.894274 6842 tablet_replica.cc:330] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96: stopping tablet replica
I20260809 12:59:49.894908 6842 raft_consensus.cc:2229] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:59:49.895089 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 6341
W20260809 12:59:49.916915 6279 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.235.65:42643 (error 108)
I20260809 12:59:49.917479 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 6511
I20260809 12:59:49.918879 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.918351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:42643 (local address 127.0.0.1:54572)
0809 12:59:49.918550 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:49.918701 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:42643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260809 12:59:49.919200 6279 catalog_manager.cc:4584] TS 8a6444b55f854350a5b10bbc6203df96 (127.21.235.65:42643): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ea02aabda4b6465f954455a3e72319c0: Network error: Client connection negotiation failed: client connection to 127.21.235.65:42643: connect: Connection refused (error 111)
I20260809 12:59:49.932350 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.918322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47550 (local address 127.21.235.66:44063)
0809 12:59:49.918519 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:49.918525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:49.918599 (+ 74us) server_negotiation.cc:407] Connection header received
0809 12:59:49.918759 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:49.918764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:49.918844 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:49.918965 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:49.920480 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.921228 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.922656 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:49.923110 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:49.923225 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:49.925949 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:49.925963 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:49.925978 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:49.926022 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:49.929471 (+ 3449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:49.930193 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:49.930201 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:49.930203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:49.930343 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:49.930843 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:49.930847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:49.930849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:49.931274 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:49.931515 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:49.931879 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 12:59:49.932054 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:59:49.933022 6645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:49.973053 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.972475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:42643 (local address 127.0.0.1:54580)
0809 12:59:49.972716 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:49.972832 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:42643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 12:59:49.974174 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:49.973750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:42643 (local address 127.0.0.1:54582)
0809 12:59:49.973919 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:49.974029 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:42643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 12:59:50.025990 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 6511
I20260809 12:59:50.034801 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:50.034124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:42643 (local address 127.0.0.1:54594)
0809 12:59:50.034426 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:50.034563 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:42643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 12:59:50.035981 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:50.035542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:42643 (local address 127.0.0.1:54610)
0809 12:59:50.035720 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:50.035819 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:42643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 12:59:50.045261 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 6654
I20260809 12:59:50.059533 6810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:50.046107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50430 (local address 127.21.235.67:38663)
0809 12:59:50.046312 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:50.046319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:50.046340 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:59:50.046603 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:50.046607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:50.046672 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:50.046780 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:50.048143 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:50.048898 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:50.050238 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:50.050726 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:50.050866 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:50.053415 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:50.053428 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:50.053445 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:50.053482 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:50.056580 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:50.057364 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:50.057374 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:50.057377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:50.057541 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:50.057931 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:50.057937 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:50.057940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:50.058305 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:50.058777 (+ 472us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:50.058943 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 12:59:50.059181 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:59:50.060258 6791 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:50.116798 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:50.116182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:42643 (local address 127.0.0.1:54620)
0809 12:59:50.116483 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:50.116607 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:42643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260809 12:59:50.117991 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:50.117553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:42643 (local address 127.0.0.1:54636)
0809 12:59:50.117768 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:50.117856 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:42643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260809 12:59:50.152017 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 6654
I20260809 12:59:50.173305 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 6262
I20260809 12:59:50.174577 6333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:59:50.294433 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:50.293352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:42643 (local address 127.0.0.1:54644)
0809 12:59:50.293562 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:50.293692 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:42643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 12:59:50.294504 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 6262
I20260809 12:59:50.320451 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:40589
--webserver_interface=127.21.235.126
--webserver_port=43615
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:40589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:59:50.660917 6850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:50.661196 6850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:50.661375 6850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:50.668483 6850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:59:50.668588 6850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:50.668689 6850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:50.668789 6850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:59:50.668840 6850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:59:50.678459 6850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:40589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:40589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=43615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:50.679785 6850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:50.681135 6850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:50.693097 6855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:50.693050 6858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:50.693132 6856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:50.694207 6850 server_base.cc:1034] running on GCE node
I20260809 12:59:50.695008 6850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:50.696316 6850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:50.697516 6850 hybrid_clock.cc:648] HybridClock initialized: now 1786280390697462 us; error 52 us; skew 500 ppm
I20260809 12:59:50.697803 6850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:50 dist-test-slave-gp6n krb5kdc[5664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280390, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:50.701018 6850 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 12:59:50.702589 6850 webserver.cc:460] Webserver started at http://127.21.235.126:43615/ using document root <none> and password file <none>
I20260809 12:59:50.703226 6850 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:50.703330 6850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:50.708688 6850 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 12:59:50.711642 6865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:50.712981 6850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:59:50.713160 6850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d"
format_stamp: "Formatted at 2026-08-09 12:59:44 on dist-test-slave-gp6n"
server_key: "01f6e554a73bfdbd04ccc1245b17a941"
server_key_iv: "fb8bdf9c3910f4eafe0d61019e067e6b"
server_key_version: "encryptionkey@0"
I20260809 12:59:50.714160 6850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:50.749653 6850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:50.753908 6850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:50.754204 6850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:50.774551 6850 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:40589
I20260809 12:59:50.774641 6924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:40589 every 8 connection(s)
I20260809 12:59:50.776113 6850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 12:59:50.776760 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 6850
I20260809 12:59:50.777793 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:42643
--local_ip_for_outbound_sockets=127.21.235.65
--tserver_master_addrs=127.21.235.126:40589
--webserver_port=37913
--webserver_interface=127.21.235.65
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:59:50.783388 6925 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:59:50.786660 6925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d: Bootstrap starting.
I20260809 12:59:50.806756 6925 log.cc:824] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:50.824286 6925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:59:50.824887 6925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d: Bootstrap complete.
I20260809 12:59:50.834223 6925 raft_consensus.cc:348] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40589 } }
I20260809 12:59:50.835186 6925 raft_consensus.cc:729] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87bf791a9d742a9b1d0bd8e4bb3134d, State: Initialized, Role: FOLLOWER
I20260809 12:59:50.835862 6925 consensus_queue.cc:260] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40589 } }
I20260809 12:59:50.836131 6925 raft_consensus.cc:388] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:59:50.836270 6925 raft_consensus.cc:482] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:59:50.836484 6925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 2 FOLLOWER]: Advancing to term 3
I20260809 12:59:50.840464 6925 raft_consensus.cc:504] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40589 } }
I20260809 12:59:50.841001 6925 leader_election.cc:302] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c87bf791a9d742a9b1d0bd8e4bb3134d; no voters:
I20260809 12:59:50.841748 6925 leader_election.cc:288] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 12:59:50.841917 6929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 3 FOLLOWER]: Leader election won for term 3
I20260809 12:59:50.843091 6929 raft_consensus.cc:686] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [term 3 LEADER]: Becoming Leader. State: Replica: c87bf791a9d742a9b1d0bd8e4bb3134d, State: Running, Role: LEADER
I20260809 12:59:50.843747 6929 consensus_queue.cc:237] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40589 } }
I20260809 12:59:50.844110 6925 sys_catalog.cc:564] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:59:50.846417 6932 sys_catalog.cc:455] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c87bf791a9d742a9b1d0bd8e4bb3134d. Latest consensus state: current_term: 3 leader_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40589 } } }
I20260809 12:59:50.846841 6932 sys_catalog.cc:458] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [sys.catalog]: This master's current role is: LEADER
I20260809 12:59:50.847219 6930 sys_catalog.cc:455] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87bf791a9d742a9b1d0bd8e4bb3134d" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40589 } } }
I20260809 12:59:50.847615 6930 sys_catalog.cc:458] T 00000000000000000000000000000000 P c87bf791a9d742a9b1d0bd8e4bb3134d [sys.catalog]: This master's current role is: LEADER
I20260809 12:59:50.857438 6939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:59:50.866994 6939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:59:50.868739 6939 catalog_manager.cc:1259] Loaded cluster ID: 5f72890e10f44d78a008594817f0213d
I20260809 12:59:50.868916 6939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 12:59:50.871542 6939 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f5bad4709ad google::LogMessage::Fail() at ??:0
@ 0x7f5bad47388a google::LogMessage::SendToLog() at ??:0
@ 0x7f5bad4704de google::LogMessage::Flush() at ??:0
@ 0x7f5bad471a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5bc2ea3512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f5bc2ea0976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5bc2f28610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f5bc2f28439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f5bb00bc345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5bb00bf91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f5bb00bf799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f5bb009aeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5bb33746db start_thread at ??:0
@ 0x7f5bac30a61f clone at ??:0
W20260809 12:59:51.173029 6927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:51.173357 6927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:51.173547 6927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:51.181253 6927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:51.181398 6927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:51.181577 6927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 12:59:51.191228 6927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:42643
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=37913
--tserver_master_addrs=127.21.235.126:40589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:51.192508 6927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:51.193835 6927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:51.207044 6970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:51.207082 6971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:51.207118 6973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:51.208021 6927 server_base.cc:1034] running on GCE node
I20260809 12:59:51.208658 6927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:51.209578 6927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:51.210784 6927 hybrid_clock.cc:648] HybridClock initialized: now 1786280391210745 us; error 52 us; skew 500 ppm
I20260809 12:59:51.211099 6927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:51 dist-test-slave-gp6n krb5kdc[5664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280391, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:51.214270 6927 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 12:59:51.215816 6927 webserver.cc:460] Webserver started at http://127.21.235.65:37913/ using document root <none> and password file <none>
I20260809 12:59:51.216552 6927 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:51.216702 6927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:51.221822 6927 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:59:51.224735 6983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:51.226032 6927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:59:51.226198 6927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "8a6444b55f854350a5b10bbc6203df96"
format_stamp: "Formatted at 2026-08-09 12:59:45 on dist-test-slave-gp6n"
server_key: "02452535ac568896337ab293a8fd4862"
server_key_iv: "c10546a5f265bfc24354472c84da4726"
server_key_version: "encryptionkey@0"
I20260809 12:59:51.227257 6927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:51.248026 6927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:51.252600 6927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:51.252910 6927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:51.253932 6927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:59:51.256393 6991 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260809 12:59:51.262302 6927 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 12:59:51.262526 6927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260809 12:59:51.262682 6927 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 12:59:51.265897 6927 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 12:59:51.266001 6927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260809 12:59:51.266355 6991 tablet_bootstrap.cc:492] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96: Bootstrap starting.
I20260809 12:59:51.293574 6991 log.cc:824] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96: Log is configured to *not* fsync() on all Append() calls
I20260809 12:59:51.306752 6991 tablet_bootstrap.cc:492] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:59:51.307548 6991 tablet_bootstrap.cc:492] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96: Bootstrap complete.
I20260809 12:59:51.309473 6991 ts_tablet_manager.cc:1397] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96: Time spent bootstrapping tablet: real 0.043s user 0.020s sys 0.019s
I20260809 12:59:51.314707 6927 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:42643
I20260809 12:59:51.314726 7101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:42643 every 8 connection(s)
I20260809 12:59:51.316156 6927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 12:59:51.316717 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 6927
I20260809 12:59:51.317520 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:44063
--local_ip_for_outbound_sockets=127.21.235.66
--tserver_master_addrs=127.21.235.126:40589
--webserver_port=46425
--webserver_interface=127.21.235.66
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:59:51.318861 6991 raft_consensus.cc:348] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } }
I20260809 12:59:51.320488 6991 raft_consensus.cc:729] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6444b55f854350a5b10bbc6203df96, State: Initialized, Role: FOLLOWER
I20260809 12:59:51.323763 6991 consensus_queue.cc:260] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } }
I20260809 12:59:51.325824 7105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:51.320179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:35231)
0809 12:59:51.322690 (+ 2511us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:51.324144 (+ 1454us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2172,"thread_start_us":1916,"threads_started":1}
W20260809 12:59:51.327035 7103 heartbeater.cc:644] Failed to heartbeat to 127.21.235.126:40589 (0 consecutive failures): Network error: Failed to ping master at 127.21.235.126:40589: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
I20260809 12:59:51.327957 7105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:51.327621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:33549)
0809 12:59:51.327776 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:51.327853 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 12:59:51.328866 7105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:51.328505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:54247)
0809 12:59:51.328655 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:51.328742 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 12:59:51.329134 7103 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 12:59:51.330859 7105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:51.330472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:49109)
0809 12:59:51.330616 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:51.330703 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 12:59:51.330988 6991 ts_tablet_manager.cc:1428] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96: Time spent starting tablet: real 0.021s user 0.011s sys 0.010s
I20260809 12:59:51.332078 7105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:51.331646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:43337)
0809 12:59:51.331835 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:51.331922 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 12:59:51.660029 7107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:51.660337 7107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:51.660508 7107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:51.667914 7107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:51.668025 7107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:51.668251 7107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 12:59:51.678057 7107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:44063
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=46425
--tserver_master_addrs=127.21.235.126:40589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:51.679415 7107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:51.680761 7107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:51.694348 7116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:51.694473 7112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:51.694473 7114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:51.695250 7107 server_base.cc:1034] running on GCE node
I20260809 12:59:51.695825 7107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:51.696764 7107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:51.698009 7107 hybrid_clock.cc:648] HybridClock initialized: now 1786280391697963 us; error 56 us; skew 500 ppm
I20260809 12:59:51.698328 7107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:51 dist-test-slave-gp6n krb5kdc[5664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280391, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:51.701383 7107 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 12:59:51.702876 7107 webserver.cc:460] Webserver started at http://127.21.235.66:46425/ using document root <none> and password file <none>
I20260809 12:59:51.703554 7107 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:51.703704 7107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:51.708818 7107 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 12:59:51.711406 7125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:51.712667 7107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:59:51.712842 7107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "ddd38535e34b4ffb863ee3126190ab3d"
format_stamp: "Formatted at 2026-08-09 12:59:45 on dist-test-slave-gp6n"
server_key: "f085f1cc929d0e379918826c255c36a8"
server_key_iv: "3ae04146b7d434443e7d91dee4ffab22"
server_key_version: "encryptionkey@0"
I20260809 12:59:51.713814 7107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:51.729642 7107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:51.733566 7107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:51.733845 7107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:51.734823 7107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:59:51.736111 7107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:59:51.736243 7107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:51.736369 7107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:59:51.736464 7107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:51.770349 7107 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:44063
I20260809 12:59:51.770576 7250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:44063 every 8 connection(s)
I20260809 12:59:51.771893 7107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 12:59:51.776662 7253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:51.774922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:59219)
0809 12:59:51.775641 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:51.775854 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":272,"threads_started":1}
W20260809 12:59:51.777285 7251 heartbeater.cc:644] Failed to heartbeat to 127.21.235.126:40589 (0 consecutive failures): Network error: Failed to ping master at 127.21.235.126:40589: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
I20260809 12:59:51.778034 7253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:51.777732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:44275)
0809 12:59:51.777865 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:51.777937 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 12:59:51.778864 7253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:51.778565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:54207)
0809 12:59:51.778674 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:51.778762 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 12:59:51.779127 7251 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 12:59:51.779244 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 7107
I20260809 12:59:51.779850 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:38663
--local_ip_for_outbound_sockets=127.21.235.67
--tserver_master_addrs=127.21.235.126:40589
--webserver_port=40971
--webserver_interface=127.21.235.67
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:59:52.115159 7254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:59:52.115482 7254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:59:52.115655 7254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:59:52.122881 7254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:59:52.123039 7254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:59:52.123211 7254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 12:59:52.133698 7254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:38663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=40971
--tserver_master_addrs=127.21.235.126:40589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 12:59:52.135156 7254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:59:52.136528 7254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:59:52.150507 7262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:52.150487 7260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:59:52.150560 7259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:59:52.151386 7254 server_base.cc:1034] running on GCE node
I20260809 12:59:52.151980 7254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:59:52.152931 7254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:59:52.154143 7254 hybrid_clock.cc:648] HybridClock initialized: now 1786280392154090 us; error 46 us; skew 500 ppm
I20260809 12:59:52.154523 7254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:59:52 dist-test-slave-gp6n krb5kdc[5664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280392, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:59:52.162248 7254 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 12:59:52.163735 7254 webserver.cc:460] Webserver started at http://127.21.235.67:40971/ using document root <none> and password file <none>
I20260809 12:59:52.164456 7254 fs_manager.cc:359] Metadata directory not provided
I20260809 12:59:52.164594 7254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:59:52.170012 7254 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:59:52.172731 7270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:52.174036 7254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:59:52.174206 7254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "b1b83dc844244333baaea9e3cb255212"
format_stamp: "Formatted at 2026-08-09 12:59:46 on dist-test-slave-gp6n"
server_key: "6eab2ef5aa3b3eeaf9629545c6605e66"
server_key_iv: "61b3a1291d8565ee5d7bcc75dc8a3bce"
server_key_version: "encryptionkey@0"
I20260809 12:59:52.175217 7254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:59:52.206931 7254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:59:52.210958 7254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:59:52.211228 7254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:59:52.212179 7254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:59:52.213487 7254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:59:52.213598 7254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:52.213730 7254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:59:52.213811 7254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:59:52.250789 7254 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:38663
I20260809 12:59:52.250881 7395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:38663 every 8 connection(s)
I20260809 12:59:52.252266 7254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 12:59:52.256897 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 7254
I20260809 12:59:52.257197 7399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:52.255250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:35253)
0809 12:59:52.256041 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:52.256254 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":281,"threads_started":1}
W20260809 12:59:52.257943 7396 heartbeater.cc:644] Failed to heartbeat to 127.21.235.126:40589 (0 consecutive failures): Network error: Failed to ping master at 127.21.235.126:40589: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
I20260809 12:59:52.258877 7399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:52.258540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:58437)
0809 12:59:52.258675 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:52.258738 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 12:59:52.259627 7399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:52.259342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:51491)
0809 12:59:52.259458 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:52.259526 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 12:59:52.259846 7396 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 12:59:52.333647 7401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:52.332870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:55459)
0809 12:59:52.333327 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:52.333429 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":229,"threads_started":1}
I20260809 12:59:52.780648 7402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:52.779648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:42555)
0809 12:59:52.780184 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:52.780313 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":296,"threads_started":1}
I20260809 12:59:52.970577 7403 raft_consensus.cc:482] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:59:52.970888 7403 raft_consensus.cc:504] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } }
I20260809 12:59:52.973129 7403 leader_election.cc:288] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1b83dc844244333baaea9e3cb255212 (127.21.235.67:38663), ddd38535e34b4ffb863ee3126190ab3d (127.21.235.66:44063)
Aug 09 12:59:52 dist-test-slave-gp6n krb5kdc[5664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280391, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.67@KRBTEST.COM
Aug 09 12:59:52 dist-test-slave-gp6n krb5kdc[5664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280391, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.66@KRBTEST.COM
I20260809 12:59:53.007369 7404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:52.973398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:38663 (local address 127.21.235.65:35109)
0809 12:59:52.974040 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:52.974084 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:52.974752 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:52.984856 (+ 10104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:52.984867 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:52.985781 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:52.986534 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:52.986556 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:52.988084 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:52.988089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:52.989549 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:52.989561 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:52.990940 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:52.990947 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:52.991132 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:52.996411 (+ 5279us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:52.996439 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:52.999192 (+ 2753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:53.003342 (+ 4150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:53.003353 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:53.003371 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:53.004009 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:53.004584 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:53.004588 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:53.004592 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:53.004813 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:53.005672 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:53.005680 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:53.006034 (+ 354us) client_negotiation.cc:769] Sending connection context
0809 12:59:53.006316 (+ 282us) client_negotiation.cc:240] Negotiation successful
0809 12:59:53.006879 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":195,"spinlock_wait_cycles":17024,"thread_start_us":319,"threads_started":1}
I20260809 12:59:53.007411 7406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:52.973628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:35109 (local address 127.21.235.67:38663)
0809 12:59:52.982562 (+ 8934us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:52.982574 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:52.982606 (+ 32us) server_negotiation.cc:407] Connection header received
0809 12:59:52.984003 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:52.984012 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:52.984359 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:52.984699 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:52.986740 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:52.987893 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:52.989691 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:52.990365 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:52.990628 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:52.999581 (+ 8953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:52.999594 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:52.999615 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:52.999675 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:53.003181 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:53.004162 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:53.004172 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:53.004180 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:53.004419 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:53.004940 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:53.004945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:53.004957 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:53.005510 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:53.005714 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:53.006589 (+ 875us) server_negotiation.cc:299] Negotiation successful
0809 12:59:53.006987 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8723,"thread_start_us":11479,"threads_started":1}
I20260809 12:59:53.007409 7407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:52.974192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:56385 (local address 127.21.235.66:44063)
0809 12:59:52.982291 (+ 8099us) server_negotiation.cc:206] Beginning negotiation
0809 12:59:52.982304 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:59:52.982345 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:59:52.983967 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:59:52.983976 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:59:52.984360 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:59:52.984708 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:59:52.986736 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:52.987885 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:52.989687 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:59:52.990311 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:59:52.990531 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:52.999895 (+ 9364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:59:52.999903 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:59:52.999915 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:59:52.999965 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:59:53.003741 (+ 3776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:53.004537 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:53.004544 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:53.004549 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:53.004720 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:59:53.005147 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:59:53.005157 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:59:53.005159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:59:53.005678 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:59:53.005883 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 12:59:53.006615 (+ 732us) server_negotiation.cc:299] Negotiation successful
0809 12:59:53.006998 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7940,"thread_start_us":7799,"threads_started":1}
I20260809 12:59:53.007480 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:52.973447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:44063 (local address 127.21.235.65:56385)
0809 12:59:52.974571 (+ 1124us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:52.974596 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:59:52.974749 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:59:52.984857 (+ 10108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:59:52.984868 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:59:52.985781 (+ 913us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:59:52.986540 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:52.986562 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:52.988099 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:52.988104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:52.989547 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:59:52.989559 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:59:52.990545 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:59:52.990551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:59:52.990798 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:59:52.996411 (+ 5613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:59:52.996451 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:59:52.999606 (+ 3155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:59:53.003892 (+ 4286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:53.003896 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:53.003898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:53.004353 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:53.004846 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:59:53.004851 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:59:53.004855 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:59:53.005023 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:59:53.005812 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:59:53.005816 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:59:53.006034 (+ 218us) client_negotiation.cc:769] Sending connection context
0809 12:59:53.006315 (+ 281us) client_negotiation.cc:240] Negotiation successful
0809 12:59:53.007045 (+ 730us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":932,"mutex_wait_us":307,"spinlock_wait_cycles":31872,"thread_start_us":973,"threads_started":1}
W20260809 12:59:53.011649 6986 leader_election.cc:341] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ddd38535e34b4ffb863ee3126190ab3d (127.21.235.66:44063): Not found: Tablet not found: ea02aabda4b6465f954455a3e72319c0
W20260809 12:59:53.011922 6987 leader_election.cc:341] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1b83dc844244333baaea9e3cb255212 (127.21.235.67:38663): Not found: Tablet not found: ea02aabda4b6465f954455a3e72319c0
I20260809 12:59:53.012120 6987 leader_election.cc:302] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6444b55f854350a5b10bbc6203df96; no voters: b1b83dc844244333baaea9e3cb255212, ddd38535e34b4ffb863ee3126190ab3d
I20260809 12:59:53.012599 7403 raft_consensus.cc:2726] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 12:59:53.261133 7408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:53.260324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:40121)
0809 12:59:53.260766 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:53.260904 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":220,"threads_started":1}
I20260809 12:59:53.335135 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:53.334617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:48431)
0809 12:59:53.334826 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:53.334927 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 12:59:53.782116 7409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:53.781467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:41317)
0809 12:59:53.781882 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:53.781963 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":226,"threads_started":1}
I20260809 12:59:54.262739 7410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:54.262016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:50975)
0809 12:59:54.262501 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:54.262587 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":225,"threads_started":1}
I20260809 12:59:54.336851 7411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:54.336066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:46001)
0809 12:59:54.336592 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:54.336674 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":233,"threads_started":1}
I20260809 12:59:54.596908 7412 raft_consensus.cc:482] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:59:54.597163 7412 raft_consensus.cc:504] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } }
I20260809 12:59:54.597966 7412 leader_election.cc:288] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1b83dc844244333baaea9e3cb255212 (127.21.235.67:38663), ddd38535e34b4ffb863ee3126190ab3d (127.21.235.66:44063)
W20260809 12:59:54.599133 6986 leader_election.cc:341] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ddd38535e34b4ffb863ee3126190ab3d (127.21.235.66:44063): Not found: Tablet not found: ea02aabda4b6465f954455a3e72319c0
W20260809 12:59:54.599352 6987 leader_election.cc:341] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1b83dc844244333baaea9e3cb255212 (127.21.235.67:38663): Not found: Tablet not found: ea02aabda4b6465f954455a3e72319c0
I20260809 12:59:54.599484 6987 leader_election.cc:302] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6444b55f854350a5b10bbc6203df96; no voters: b1b83dc844244333baaea9e3cb255212, ddd38535e34b4ffb863ee3126190ab3d
I20260809 12:59:54.599725 7412 raft_consensus.cc:2726] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 12:59:54.783600 7413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:54.782928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:44737)
0809 12:59:54.783344 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:54.783424 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":223,"threads_started":1}
I20260809 12:59:55.264299 7415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:55.263590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:43001)
0809 12:59:55.264049 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:55.264131 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":242,"threads_started":1}
I20260809 12:59:55.338557 7416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:55.337736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:52689)
0809 12:59:55.338202 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:55.338300 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":238,"threads_started":1}
I20260809 12:59:55.785305 7417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:55.784590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:49739)
0809 12:59:55.785034 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:55.785119 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":221,"threads_started":1}
I20260809 12:59:56.265990 7418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:56.265167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:52491)
0809 12:59:56.265717 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:56.265793 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":336,"threads_started":1}
I20260809 12:59:56.340474 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:56.339584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:56523)
0809 12:59:56.340132 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:56.340246 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":290,"threads_started":1}
I20260809 12:59:56.547811 7420 raft_consensus.cc:482] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:59:56.548044 7420 raft_consensus.cc:504] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } }
I20260809 12:59:56.548755 7420 leader_election.cc:288] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1b83dc844244333baaea9e3cb255212 (127.21.235.67:38663), ddd38535e34b4ffb863ee3126190ab3d (127.21.235.66:44063)
W20260809 12:59:56.549865 6987 leader_election.cc:341] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1b83dc844244333baaea9e3cb255212 (127.21.235.67:38663): Not found: Tablet not found: ea02aabda4b6465f954455a3e72319c0
W20260809 12:59:56.550084 6986 leader_election.cc:341] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ddd38535e34b4ffb863ee3126190ab3d (127.21.235.66:44063): Not found: Tablet not found: ea02aabda4b6465f954455a3e72319c0
I20260809 12:59:56.550239 6986 leader_election.cc:302] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6444b55f854350a5b10bbc6203df96; no voters: b1b83dc844244333baaea9e3cb255212, ddd38535e34b4ffb863ee3126190ab3d
I20260809 12:59:56.550566 7420 raft_consensus.cc:2726] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 12:59:56.787088 7421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:56.786200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:40221)
0809 12:59:56.786831 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:56.786915 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":419,"threads_started":1}
I20260809 12:59:57.267751 7422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:57.266977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:42403)
0809 12:59:57.267483 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:57.267576 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":275,"threads_started":1}
I20260809 12:59:57.342164 7423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:57.341414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:49555)
0809 12:59:57.341875 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:57.341991 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":249,"threads_started":1}
I20260809 12:59:57.788713 7424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:57.788006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:52451)
0809 12:59:57.788452 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:57.788531 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":216,"threads_started":1}
I20260809 12:59:58.269451 7425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:58.268634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:57645)
0809 12:59:58.269131 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:58.269216 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":268,"threads_started":1}
I20260809 12:59:58.343797 7426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:58.343045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:51919)
0809 12:59:58.343511 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:58.343593 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":240,"threads_started":1}
I20260809 12:59:58.790586 7427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:58.789615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:43597)
0809 12:59:58.790187 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:58.790286 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":284,"threads_started":1}
I20260809 12:59:59.271320 7428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:59.270526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:50527)
0809 12:59:59.271037 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:59.271125 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":235,"threads_started":1}
I20260809 12:59:59.345515 7429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:59.344778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:42749)
0809 12:59:59.345251 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:59.345350 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":247,"threads_started":1}
I20260809 12:59:59.792282 7430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:59:59.791570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:46383)
0809 12:59:59.792007 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 12:59:59.792108 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":239,"threads_started":1}
I20260809 13:00:00.272845 7432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:00.272190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:46239)
0809 13:00:00.272589 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:00.272666 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":210,"threads_started":1}
I20260809 13:00:00.347697 7434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:00.346822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:45389)
0809 13:00:00.347413 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:00.347505 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":350,"threads_started":1}
I20260809 13:00:00.794000 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:00.793237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:38047)
0809 13:00:00.793728 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:00.793821 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":234,"threads_started":1}
I20260809 13:00:01.274623 7436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:01.273728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:60363)
0809 13:00:01.274265 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:01.274363 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":264,"threads_started":1}
I20260809 13:00:01.349624 7437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:01.348855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:50919)
0809 13:00:01.349354 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:01.349446 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":258,"threads_started":1}
I20260809 13:00:01.473459 7438 raft_consensus.cc:482] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:00:01.473719 7438 raft_consensus.cc:504] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1b83dc844244333baaea9e3cb255212" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 38663 } } peers { permanent_uuid: "8a6444b55f854350a5b10bbc6203df96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 42643 } } peers { permanent_uuid: "ddd38535e34b4ffb863ee3126190ab3d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 44063 } }
I20260809 13:00:01.474525 7438 leader_election.cc:288] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1b83dc844244333baaea9e3cb255212 (127.21.235.67:38663), ddd38535e34b4ffb863ee3126190ab3d (127.21.235.66:44063)
W20260809 13:00:01.475512 6987 leader_election.cc:341] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1b83dc844244333baaea9e3cb255212 (127.21.235.67:38663): Not found: Tablet not found: ea02aabda4b6465f954455a3e72319c0
W20260809 13:00:01.475718 6986 leader_election.cc:341] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ddd38535e34b4ffb863ee3126190ab3d (127.21.235.66:44063): Not found: Tablet not found: ea02aabda4b6465f954455a3e72319c0
I20260809 13:00:01.475867 6986 leader_election.cc:302] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6444b55f854350a5b10bbc6203df96; no voters: b1b83dc844244333baaea9e3cb255212, ddd38535e34b4ffb863ee3126190ab3d
I20260809 13:00:01.476135 7438 raft_consensus.cc:2726] T ea02aabda4b6465f954455a3e72319c0 P 8a6444b55f854350a5b10bbc6203df96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 13:00:01.795809 7439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:01.795073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:55307)
0809 13:00:01.795537 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:01.795625 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":256,"threads_started":1}
I20260809 13:00:02.276408 7440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:02.275643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:49037)
0809 13:00:02.276127 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:02.276215 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":268,"threads_started":1}
I20260809 13:00:02.351516 7441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:02.350726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:37849)
0809 13:00:02.351232 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:02.351324 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":247,"threads_started":1}
I20260809 13:00:02.797643 7442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:02.796846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:43157)
0809 13:00:02.797384 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:02.797469 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":272,"threads_started":1}
I20260809 13:00:03.278096 7443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:03.277370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:55717)
0809 13:00:03.277821 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:03.277930 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":229,"threads_started":1}
I20260809 13:00:03.353536 7444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:03.352597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:56131)
0809 13:00:03.353169 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:03.353275 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":340,"threads_started":1}
I20260809 13:00:03.799463 7446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:03.798642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:48539)
0809 13:00:03.799151 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:03.799243 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":273,"threads_started":1}
I20260809 13:00:04.279945 7447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:04.279092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:33153)
0809 13:00:04.279688 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:04.279768 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":344,"threads_started":1}
I20260809 13:00:04.355348 7448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:04.354601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:42369)
0809 13:00:04.355076 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:04.355162 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":251,"threads_started":1}
I20260809 13:00:04.801167 7449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:04.800414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:38283)
0809 13:00:04.800897 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:04.801002 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":267,"threads_started":1}
I20260809 13:00:05.281528 7450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:05.280812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:51961)
0809 13:00:05.281239 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:05.281329 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":221,"threads_started":1}
I20260809 13:00:05.357173 7451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:05.356330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:51153)
0809 13:00:05.356894 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:05.356978 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":355,"threads_started":1}
I20260809 13:00:05.802893 7452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:05.802039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:51469)
0809 13:00:05.802572 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:05.802656 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":238,"threads_started":1}
I20260809 13:00:06.283355 7453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:06.282577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:49737)
0809 13:00:06.283055 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:06.283155 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":240,"threads_started":1}
I20260809 13:00:06.359362 7454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:06.358606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.65:41839)
0809 13:00:06.359098 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:06.359184 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":222,"threads_started":1}
I20260809 13:00:06.804862 7455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:06.804057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.66:54639)
0809 13:00:06.804559 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:06.804690 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":242,"threads_started":1}
I20260809 13:00:07.263764 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 6927
I20260809 13:00:07.280648 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:07.265159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46028 (local address 127.21.235.65:42643)
0809 13:00:07.265885 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:07.265899 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:07.265933 (+ 34us) server_negotiation.cc:407] Connection header received
0809 13:00:07.266025 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:07.266036 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:07.266343 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:07.266548 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:07.268568 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:07.269532 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:07.270939 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:07.271507 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:07.271617 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:07.274544 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:07.274556 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:07.274570 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:07.274603 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:07.277705 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:07.278493 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:07.278501 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:07.278504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:07.278639 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:07.279049 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:07.279052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:07.279054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:07.279461 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:07.279699 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:07.280017 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 13:00:07.280226 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":354,"threads_started":1}
I20260809 13:00:07.281893 7097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:00:07.285795 7458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:07.284766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40589 (local address 127.21.235.67:43463)
0809 13:00:07.285373 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:07.285523 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":277,"threads_started":1}
I20260809 13:00:07.378623 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 6927
I20260809 13:00:07.400333 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 7107
I20260809 13:00:07.419639 7460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:07.401082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44458 (local address 127.21.235.66:44063)
0809 13:00:07.401645 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:07.401650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:07.401665 (+ 15us) server_negotiation.cc:407] Connection header received
0809 13:00:07.401723 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:07.401727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:07.401794 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:07.401930 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:07.403426 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:07.404247 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:07.405672 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:07.406106 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:07.406253 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:07.413364 (+ 7111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:07.413371 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:07.413375 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:07.413420 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:07.416478 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:07.417351 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:07.417357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:07.417361 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:07.417523 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:07.417941 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:07.417950 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:07.417952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:07.418269 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:07.418622 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:07.418924 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 13:00:07.419175 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":271,"threads_started":1}
I20260809 13:00:07.420428 7246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:00:07.506289 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 7107
I20260809 13:00:07.528127 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 7254
I20260809 13:00:07.541935 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:07.529058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40224 (local address 127.21.235.67:38663)
0809 13:00:07.529692 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:07.529696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:07.529715 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:00:07.529773 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:07.529777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:07.529836 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:07.529932 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:07.531485 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:07.532285 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:07.533510 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:07.533926 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:07.534024 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:07.536890 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:07.536896 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:07.536899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:07.536934 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:07.539619 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:07.540314 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:07.540318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:07.540320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:07.540467 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:07.540789 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:07.540792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:07.540793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:07.541038 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:07.541250 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:07.541464 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 13:00:07.541630 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":354,"threads_started":1}
I20260809 13:00:07.542666 7391 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:00:07.626991 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 7254
2026-08-09T13:00:07Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24748 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:00:07 dist-test-slave-gp6n krb5kdc[7467](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:00:07 dist-test-slave-gp6n krb5kdc[7467](info): set up 2 sockets
Aug 09 13:00:07 dist-test-slave-gp6n krb5kdc[7467](info): commencing operation
krb5kdc: starting...
W20260809 13:00:08.881771 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:00:09 dist-test-slave-gp6n krb5kdc[7467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:00:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:00:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 13:00:09.189875 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:42617
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:46303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:42617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:00:09.541417 7484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:00:09.541803 7484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:00:09.541973 7484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:00:09.549652 7484 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 13:00:09.549808 7484 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 13:00:09.549921 7484 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 13:00:09.550017 7484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:00:09.550123 7484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:00:09.550248 7484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:00:09.550339 7484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:00:09.550482 7484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:00:09.560904 7484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:46303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:42617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:42617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:00:09.562316 7484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:00:09.563725 7484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:00:09.576150 7489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:09.576205 7490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:09.576292 7493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:00:09.576892 7484 server_base.cc:1034] running on GCE node
I20260809 13:00:09.577736 7484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:00:09.579279 7484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:00:09.580514 7484 hybrid_clock.cc:648] HybridClock initialized: now 1786280409580449 us; error 34 us; skew 500 ppm
I20260809 13:00:09.580875 7484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:00:09 dist-test-slave-gp6n krb5kdc[7467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280409, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:00:09.584394 7484 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 13:00:09.585942 7484 webserver.cc:460] Webserver started at http://127.21.235.126:40127/ using document root <none> and password file <none>
I20260809 13:00:09.586746 7484 fs_manager.cc:359] Metadata directory not provided
I20260809 13:00:09.586890 7484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:00:09.587251 7484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:00:09.590476 7484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "6b7e946c4e184052afd2f872047d8e42"
format_stamp: "Formatted at 2026-08-09 13:00:09 on dist-test-slave-gp6n"
server_key: "12c95c171f36288dc31c1a6cc4842007"
server_key_iv: "2ec67b21f6cf591f52199be3f9c4f3ce"
server_key_version: "encryptionkey@0"
I20260809 13:00:09.591336 7484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "6b7e946c4e184052afd2f872047d8e42"
format_stamp: "Formatted at 2026-08-09 13:00:09 on dist-test-slave-gp6n"
server_key: "12c95c171f36288dc31c1a6cc4842007"
server_key_iv: "2ec67b21f6cf591f52199be3f9c4f3ce"
server_key_version: "encryptionkey@0"
I20260809 13:00:09.596658 7484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:00:09.600189 7500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:09.601500 7484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:00:09.601686 7484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "6b7e946c4e184052afd2f872047d8e42"
format_stamp: "Formatted at 2026-08-09 13:00:09 on dist-test-slave-gp6n"
server_key: "12c95c171f36288dc31c1a6cc4842007"
server_key_iv: "2ec67b21f6cf591f52199be3f9c4f3ce"
server_key_version: "encryptionkey@0"
I20260809 13:00:09.601923 7484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:00:09.614859 7484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:00:09.619058 7484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:00:09.619340 7484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:00:09.639002 7484 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:42617
I20260809 13:00:09.639036 7554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:42617 every 8 connection(s)
I20260809 13:00:09.640393 7484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:00:09.644246 7555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:00:09.650329 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 7484
I20260809 13:00:09.650586 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:00:09.651504 22445 external_mini_cluster.cc:1428] Setting key 38e3763d351c02a7e9363046eeae0a2d
I20260809 13:00:09.656152 7555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: Bootstrap starting.
I20260809 13:00:09.659401 7555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: Neither blocks nor log segments found. Creating new log.
I20260809 13:00:09.660511 7555 log.cc:824] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: Log is configured to *not* fsync() on all Append() calls
I20260809 13:00:09.663668 7555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: No bootstrap required, opened a new log
Aug 09 13:00:09 dist-test-slave-gp6n krb5kdc[7467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:00:09.673362 7555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7e946c4e184052afd2f872047d8e42" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42617 } }
I20260809 13:00:09.673808 7555 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:00:09.673946 7555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b7e946c4e184052afd2f872047d8e42, State: Initialized, Role: FOLLOWER
I20260809 13:00:09.674750 7555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7e946c4e184052afd2f872047d8e42" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42617 } }
I20260809 13:00:09.675046 7555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:00:09.675230 7555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:00:09.675422 7555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:00:09.676153 7558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:09.653794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33460 (local address 127.21.235.126:42617)
0809 13:00:09.654689 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:09.654702 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:09.654743 (+ 41us) server_negotiation.cc:407] Connection header received
0809 13:00:09.656533 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:09.656556 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:09.656960 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:09.657346 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:09.658963 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:09.660345 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:09.662119 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:09.662919 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:09.663123 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:09.667973 (+ 4850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:09.667981 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:09.667998 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:09.668037 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:09.671491 (+ 3454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:09.672405 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:09.672411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:09.672419 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:09.672584 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:09.673029 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:09.673033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:09.673035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:09.673591 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:09.673891 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:09.674421 (+ 530us) server_negotiation.cc:299] Negotiation successful
0809 13:00:09.674858 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":364,"threads_started":1}
I20260809 13:00:09.677989 7555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7e946c4e184052afd2f872047d8e42" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42617 } }
I20260809 13:00:09.678581 7555 leader_election.cc:302] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b7e946c4e184052afd2f872047d8e42; no voters:
I20260809 13:00:09.679280 7555 leader_election.cc:288] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:00:09.679445 7561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:00:09.680275 7561 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 1 LEADER]: Becoming Leader. State: Replica: 6b7e946c4e184052afd2f872047d8e42, State: Running, Role: LEADER
I20260809 13:00:09.680948 7561 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7e946c4e184052afd2f872047d8e42" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42617 } }
I20260809 13:00:09.681927 7555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:00:09.684446 7564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b7e946c4e184052afd2f872047d8e42. Latest consensus state: current_term: 1 leader_uuid: "6b7e946c4e184052afd2f872047d8e42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7e946c4e184052afd2f872047d8e42" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42617 } } }
I20260809 13:00:09.684445 7563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b7e946c4e184052afd2f872047d8e42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7e946c4e184052afd2f872047d8e42" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42617 } } }
I20260809 13:00:09.684947 7564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [sys.catalog]: This master's current role is: LEADER
I20260809 13:00:09.684978 7563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [sys.catalog]: This master's current role is: LEADER
I20260809 13:00:09.686756 7567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:00:09.691905 7567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:00:09.702067 7567 catalog_manager.cc:1347] Generated new cluster ID: a31abf667e9f4c52aebd5855c2b155f8
I20260809 13:00:09.702230 7567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:00:09.718657 7567 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:00:09.720394 7567 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:00:09.737756 7567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: Generated new TSK 0
I20260809 13:00:09.738873 7567 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
I20260809 13:00:09.827049 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:42617
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:46303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:00:10.180608 7584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:00:10.180929 7584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:00:10.181030 7584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:00:10.188802 7584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:00:10.188933 7584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:00:10.189052 7584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 13:00:10.200160 7584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:46303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:42617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:00:10.201504 7584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:00:10.202937 7584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:00:10.216545 7591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:10.216639 7590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:10.216696 7593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:00:10.218080 7584 server_base.cc:1034] running on GCE node
I20260809 13:00:10.218904 7584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:00:10.220201 7584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:00:10.221488 7584 hybrid_clock.cc:648] HybridClock initialized: now 1786280410221396 us; error 55 us; skew 500 ppm
I20260809 13:00:10.221926 7584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:00:10 dist-test-slave-gp6n krb5kdc[7467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280410, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:00:10.225528 7584 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 13:00:10.226953 7584 webserver.cc:460] Webserver started at http://127.21.235.65:45979/ using document root <none> and password file <none>
I20260809 13:00:10.227600 7584 fs_manager.cc:359] Metadata directory not provided
I20260809 13:00:10.227703 7584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:00:10.227950 7584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:00:10.230887 7584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "f99bf84f74b948159d5dfebf961c10ee"
format_stamp: "Formatted at 2026-08-09 13:00:10 on dist-test-slave-gp6n"
server_key: "c6727ef2bc6b1b70e3a1d3451fe8116f"
server_key_iv: "d84bf348ff324e47609313cef9a57d41"
server_key_version: "encryptionkey@0"
I20260809 13:00:10.231607 7584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "f99bf84f74b948159d5dfebf961c10ee"
format_stamp: "Formatted at 2026-08-09 13:00:10 on dist-test-slave-gp6n"
server_key: "c6727ef2bc6b1b70e3a1d3451fe8116f"
server_key_iv: "d84bf348ff324e47609313cef9a57d41"
server_key_version: "encryptionkey@0"
I20260809 13:00:10.236629 7584 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:00:10.240149 7601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:10.241573 7584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:00:10.241747 7584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "f99bf84f74b948159d5dfebf961c10ee"
format_stamp: "Formatted at 2026-08-09 13:00:10 on dist-test-slave-gp6n"
server_key: "c6727ef2bc6b1b70e3a1d3451fe8116f"
server_key_iv: "d84bf348ff324e47609313cef9a57d41"
server_key_version: "encryptionkey@0"
I20260809 13:00:10.241950 7584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:00:10.258976 7584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:00:10.263445 7584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:00:10.263752 7584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:00:10.264833 7584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:00:10.266219 7584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:00:10.266294 7584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:10.266456 7584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:00:10.266512 7584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 13:00:10.302667 7584 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:41273
I20260809 13:00:10.302712 7727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:41273 every 8 connection(s)
I20260809 13:00:10.304062 7584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 13:00:10.306246 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 7584
I20260809 13:00:10.306501 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 13:00:10.307346 22445 external_mini_cluster.cc:1428] Setting key ec5854d89641315ac98bf96f35c23b45
Aug 09 13:00:10 dist-test-slave-gp6n krb5kdc[7467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280410, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:00:10.330088 7732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:10.307133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:46065 (local address 127.21.235.126:42617)
0809 13:00:10.307679 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:10.307686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:10.308871 (+ 1185us) server_negotiation.cc:407] Connection header received
0809 13:00:10.311622 (+ 2751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:10.311626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:10.311691 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:10.311801 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:10.313992 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:10.314920 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:10.317521 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:10.317980 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:10.318095 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:10.323201 (+ 5106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:10.323208 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:10.323212 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:10.323257 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:10.326659 (+ 3402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:10.327636 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:10.327640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:10.327642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:10.327770 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:10.328286 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:10.328289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:10.328291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:10.328547 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:10.328659 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:10.329620 (+ 961us) server_negotiation.cc:299] Negotiation successful
0809 13:00:10.329799 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":276,"threads_started":1}
I20260809 13:00:10.331827 7733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:10.307732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:46065)
0809 13:00:10.308702 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:10.308744 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 13:00:10.311292 (+ 2548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:00:10.312091 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:00:10.312106 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:00:10.312916 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:00:10.313790 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:10.313818 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:10.315105 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:10.315113 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:10.317353 (+ 2240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:10.317372 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:10.318235 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:10.318245 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:10.318631 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:10.319418 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:00:10.319453 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:00:10.322894 (+ 3441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:00:10.326812 (+ 3918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:00:10.326832 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:00:10.326852 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:00:10.327506 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:00:10.327895 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:00:10.327902 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:00:10.327909 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:00:10.328176 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:00:10.328686 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:00:10.328697 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:00:10.329175 (+ 478us) client_negotiation.cc:769] Sending connection context
0809 13:00:10.329512 (+ 337us) client_negotiation.cc:240] Negotiation successful
0809 13:00:10.330179 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":400,"threads_started":1}
I20260809 13:00:10.333544 7728 heartbeater.cc:347] Connected to a master server at 127.21.235.126:42617
I20260809 13:00:10.333878 7728 heartbeater.cc:464] Registering TS with master...
I20260809 13:00:10.334690 7728 heartbeater.cc:511] Master 127.21.235.126:42617 requested a full tablet report, sending...
I20260809 13:00:10.336838 7518 ts_manager.cc:194] Registered new tserver with Master: f99bf84f74b948159d5dfebf961c10ee (127.21.235.65:41273)
I20260809 13:00:10.338292 7518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:46065
WARNING: no policy specified for kudu/127.21.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
I20260809 13:00:10.377863 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:42617
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:46303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:00:10.721177 7736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:00:10.721482 7736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:00:10.721580 7736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:00:10.728780 7736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:00:10.728892 7736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:00:10.729005 7736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 13:00:10.738842 7736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:46303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:42617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:00:10.740094 7736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:00:10.741444 7736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:00:10.755419 7744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:10.755369 7741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:10.755467 7742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:00:10.760030 7736 server_base.cc:1034] running on GCE node
I20260809 13:00:10.760708 7736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:00:10.761713 7736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:00:10.762940 7736 hybrid_clock.cc:648] HybridClock initialized: now 1786280410762893 us; error 43 us; skew 500 ppm
I20260809 13:00:10.763275 7736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:00:10 dist-test-slave-gp6n krb5kdc[7467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280410, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:00:10.766625 7736 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 13:00:10.768252 7736 webserver.cc:460] Webserver started at http://127.21.235.66:37971/ using document root <none> and password file <none>
I20260809 13:00:10.768983 7736 fs_manager.cc:359] Metadata directory not provided
I20260809 13:00:10.769126 7736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:00:10.769451 7736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:00:10.772588 7736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "929b08811b7e4e91b3cec796420a52ef"
format_stamp: "Formatted at 2026-08-09 13:00:10 on dist-test-slave-gp6n"
server_key: "b6ba40033a26cf8ec5130ffb473161ab"
server_key_iv: "dfedc33a3e3196d88210cc82350c8c7c"
server_key_version: "encryptionkey@0"
I20260809 13:00:10.773396 7736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "929b08811b7e4e91b3cec796420a52ef"
format_stamp: "Formatted at 2026-08-09 13:00:10 on dist-test-slave-gp6n"
server_key: "b6ba40033a26cf8ec5130ffb473161ab"
server_key_iv: "dfedc33a3e3196d88210cc82350c8c7c"
server_key_version: "encryptionkey@0"
I20260809 13:00:10.778779 7736 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 13:00:10.782258 7751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:10.783777 7736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:00:10.783946 7736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "929b08811b7e4e91b3cec796420a52ef"
format_stamp: "Formatted at 2026-08-09 13:00:10 on dist-test-slave-gp6n"
server_key: "b6ba40033a26cf8ec5130ffb473161ab"
server_key_iv: "dfedc33a3e3196d88210cc82350c8c7c"
server_key_version: "encryptionkey@0"
I20260809 13:00:10.784161 7736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:00:10.809103 7736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:00:10.813694 7736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:00:10.814023 7736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:00:10.815142 7736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:00:10.816545 7736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:00:10.816658 7736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:10.816810 7736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:00:10.816883 7736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:10.851418 7736 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:46257
I20260809 13:00:10.851452 7874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:46257 every 8 connection(s)
I20260809 13:00:10.852782 7736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 13:00:10.857411 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 7736
I20260809 13:00:10.857637 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 13:00:10.858119 22445 external_mini_cluster.cc:1428] Setting key 9c906a29100ce5a4ef3925d16d1b4b81
Aug 09 13:00:10 dist-test-slave-gp6n krb5kdc[7467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280410, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:00:10.875136 7877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:10.855499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:36481 (local address 127.21.235.126:42617)
0809 13:00:10.855977 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:10.855981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:10.856834 (+ 853us) server_negotiation.cc:407] Connection header received
0809 13:00:10.859064 (+ 2230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:10.859072 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:10.859159 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:10.859296 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:10.861189 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:10.861919 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:10.863658 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:10.864132 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:10.864267 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:10.867794 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:10.867799 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:10.867803 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:10.867836 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:10.871207 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:10.872242 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:10.872250 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:10.872256 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:10.872420 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:10.872912 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:10.872919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:10.872925 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:10.873308 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:10.873462 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:10.874317 (+ 855us) server_negotiation.cc:299] Negotiation successful
0809 13:00:10.874609 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":261,"threads_started":1}
I20260809 13:00:10.875851 7878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:10.855927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:36481)
0809 13:00:10.856661 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:10.856708 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 13:00:10.858677 (+ 1969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:00:10.859591 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:00:10.859603 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:00:10.860377 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:00:10.861016 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:10.861035 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:10.862135 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:10.862147 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:10.863510 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:10.863523 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:10.864376 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:10.864383 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:10.864632 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:10.865346 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:00:10.865369 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:00:10.867557 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:00:10.871356 (+ 3799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:00:10.871368 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:00:10.871390 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:00:10.872090 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:00:10.872569 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:00:10.872572 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:00:10.872575 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:00:10.872773 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:00:10.873441 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:00:10.873446 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:00:10.873928 (+ 482us) client_negotiation.cc:769] Sending connection context
0809 13:00:10.874176 (+ 248us) client_negotiation.cc:240] Negotiation successful
0809 13:00:10.874666 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":295,"threads_started":1}
I20260809 13:00:10.877478 7875 heartbeater.cc:347] Connected to a master server at 127.21.235.126:42617
I20260809 13:00:10.877837 7875 heartbeater.cc:464] Registering TS with master...
I20260809 13:00:10.878736 7875 heartbeater.cc:511] Master 127.21.235.126:42617 requested a full tablet report, sending...
I20260809 13:00:10.880439 7518 ts_manager.cc:194] Registered new tserver with Master: 929b08811b7e4e91b3cec796420a52ef (127.21.235.66:46257)
I20260809 13:00:10.881284 7518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:36481
WARNING: no policy specified for kudu/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
I20260809 13:00:10.926633 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:42617
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:46303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:00:11.299299 7882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:00:11.299645 7882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:00:11.299815 7882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:00:11.310266 7882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:00:11.310451 7882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:00:11.310632 7882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 13:00:11.320609 7882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:46303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:42617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:00:11.321952 7882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:00:11.323344 7882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:00:11.337528 7887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:11.337634 7888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:11.337605 7890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:00:11.338531 7882 server_base.cc:1034] running on GCE node
I20260809 13:00:11.339303 7882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:00:11.340396 7882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:00:11.341351 7728 heartbeater.cc:503] Master 127.21.235.126:42617 was elected leader, sending a full tablet report...
I20260809 13:00:11.341658 7882 hybrid_clock.cc:648] HybridClock initialized: now 1786280411341582 us; error 45 us; skew 500 ppm
I20260809 13:00:11.342058 7882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:00:11 dist-test-slave-gp6n krb5kdc[7467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280411, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:00:11.345850 7882 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 13:00:11.347507 7882 webserver.cc:460] Webserver started at http://127.21.235.67:37681/ using document root <none> and password file <none>
I20260809 13:00:11.348228 7882 fs_manager.cc:359] Metadata directory not provided
I20260809 13:00:11.348333 7882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:00:11.348639 7882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:00:11.351831 7882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "390bb3a6078d436898a165c5dd8c8377"
format_stamp: "Formatted at 2026-08-09 13:00:11 on dist-test-slave-gp6n"
server_key: "6c662aa6a0db5f02d02b1fc92309573d"
server_key_iv: "766ef75f8bd3ad689e67ae4e325729c3"
server_key_version: "encryptionkey@0"
I20260809 13:00:11.352633 7882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "390bb3a6078d436898a165c5dd8c8377"
format_stamp: "Formatted at 2026-08-09 13:00:11 on dist-test-slave-gp6n"
server_key: "6c662aa6a0db5f02d02b1fc92309573d"
server_key_iv: "766ef75f8bd3ad689e67ae4e325729c3"
server_key_version: "encryptionkey@0"
I20260809 13:00:11.358186 7882 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 13:00:11.361868 7898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:11.363330 7882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:00:11.363538 7882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "390bb3a6078d436898a165c5dd8c8377"
format_stamp: "Formatted at 2026-08-09 13:00:11 on dist-test-slave-gp6n"
server_key: "6c662aa6a0db5f02d02b1fc92309573d"
server_key_iv: "766ef75f8bd3ad689e67ae4e325729c3"
server_key_version: "encryptionkey@0"
I20260809 13:00:11.363749 7882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:00:11.380457 7882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:00:11.385142 7882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:00:11.385485 7882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:00:11.386714 7882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:00:11.388218 7882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:00:11.388325 7882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:11.388473 7882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:00:11.388549 7882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:11.423635 7882 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:44887
I20260809 13:00:11.423671 8025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:44887 every 8 connection(s)
I20260809 13:00:11.424962 7882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 13:00:11.427207 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 7882
I20260809 13:00:11.427449 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 13:00:11.428081 22445 external_mini_cluster.cc:1428] Setting key 464c008c8af17528fa0135e309237d17
Aug 09 13:00:11 dist-test-slave-gp6n krb5kdc[7467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280411, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:00:11.446458 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:11.427735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:34519 (local address 127.21.235.126:42617)
0809 13:00:11.428223 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:11.428230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:11.429381 (+ 1151us) server_negotiation.cc:407] Connection header received
0809 13:00:11.431842 (+ 2461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:11.431847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:11.431947 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:11.432073 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:11.433620 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.434487 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.436070 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.436485 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.436621 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:11.440331 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:11.440336 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:11.440339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:11.440381 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:11.443310 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:11.444181 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:11.444184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:11.444187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:11.444347 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:11.444726 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:11.444730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:11.444752 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:11.445051 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:11.445173 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:11.445937 (+ 764us) server_negotiation.cc:299] Negotiation successful
0809 13:00:11.446119 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":238,"threads_started":1}
I20260809 13:00:11.447440 8032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:11.428341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:34519)
0809 13:00:11.429181 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:11.429237 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 13:00:11.431553 (+ 2316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:00:11.432322 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:00:11.432331 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:00:11.432883 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:00:11.433456 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:11.433474 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.434641 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.434646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:11.435919 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:11.435931 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.436714 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.436722 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:11.436994 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:11.437734 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:00:11.437757 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:00:11.440120 (+ 2363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:00:11.443430 (+ 3310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:00:11.443438 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:00:11.443456 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:00:11.444079 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:00:11.444446 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:00:11.444449 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:00:11.444451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:00:11.444636 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:00:11.445165 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:00:11.445171 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:00:11.445516 (+ 345us) client_negotiation.cc:769] Sending connection context
0809 13:00:11.445812 (+ 296us) client_negotiation.cc:240] Negotiation successful
0809 13:00:11.446231 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":342,"threads_started":1}
I20260809 13:00:11.448875 8026 heartbeater.cc:347] Connected to a master server at 127.21.235.126:42617
I20260809 13:00:11.449160 8026 heartbeater.cc:464] Registering TS with master...
I20260809 13:00:11.449937 8026 heartbeater.cc:511] Master 127.21.235.126:42617 requested a full tablet report, sending...
I20260809 13:00:11.451635 7518 ts_manager.cc:194] Registered new tserver with Master: 390bb3a6078d436898a165c5dd8c8377 (127.21.235.67:44887)
I20260809 13:00:11.452495 7518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:34519
I20260809 13:00:11.456434 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:00:11.475347 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:11.461748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58016 (local address 127.21.235.126:42617)
0809 13:00:11.461962 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:11.461968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:11.462500 (+ 532us) server_negotiation.cc:407] Connection header received
0809 13:00:11.462748 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:11.462753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:11.462841 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:11.462940 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:11.464467 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.465199 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.466653 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.467066 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.467169 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:11.469817 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:11.469822 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:11.469825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:11.469858 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:11.472593 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:11.473303 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:11.473308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:11.473310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:11.473443 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:11.473880 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:11.473883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:11.473885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:11.474218 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:11.474538 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:11.474897 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 13:00:11.475090 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 13:00:11.479688 7518 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58016:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:00:11.487021 7518 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:00:11.511389 8043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:11.500978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40858 (local address 127.21.235.65:41273)
0809 13:00:11.502760 (+ 1782us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:11.502766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:11.502789 (+ 23us) server_negotiation.cc:407] Connection header received
0809 13:00:11.502877 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:11.502883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:11.503174 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:11.503451 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:00:11.504203 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.506304 (+ 2101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.508723 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.510178 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.510344 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:11.510623 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:11.510862 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 13:00:11.511029 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1619,"thread_start_us":1423,"threads_started":1}
I20260809 13:00:11.511504 8042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:11.500845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:41273 (local address 127.0.0.1:40858)
0809 13:00:11.501803 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:11.501857 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 13:00:11.502042 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:00:11.503473 (+ 1431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:00:11.503479 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:00:11.503513 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:00:11.503964 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:11.503984 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.506502 (+ 2518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.506507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:11.508522 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:11.508538 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.510373 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.510380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:11.510594 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:11.510649 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 13:00:11.510803 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 13:00:11.511018 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":65,"thread_start_us":543,"threads_started":1}
I20260809 13:00:11.512072 8045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:11.502021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58284 (local address 127.21.235.66:46257)
0809 13:00:11.502666 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:11.502676 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:11.503280 (+ 604us) server_negotiation.cc:407] Connection header received
0809 13:00:11.503389 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:11.503400 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:11.503639 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:11.503841 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:00:11.504799 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.506334 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.508845 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.510353 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.510503 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:11.510718 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:11.511599 (+ 881us) server_negotiation.cc:299] Negotiation successful
0809 13:00:11.511763 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":344,"threads_started":1}
I20260809 13:00:11.512538 8044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:11.501374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:46257 (local address 127.0.0.1:58284)
0809 13:00:11.502991 (+ 1617us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:11.503010 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:00:11.503150 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:00:11.504089 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:00:11.504096 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:00:11.504133 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:00:11.504586 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:11.504599 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.506558 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.506568 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:11.508662 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:11.508679 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.510901 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.510908 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:11.511096 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:11.511129 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 13:00:11.511226 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 13:00:11.511367 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1487,"thread_start_us":1543,"threads_started":1}
I20260809 13:00:11.513193 8046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:11.503274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:44887 (local address 127.0.0.1:39610)
0809 13:00:11.504147 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:11.504167 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:00:11.504289 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:00:11.505419 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:00:11.505424 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:00:11.505449 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:00:11.505867 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:11.505879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.508067 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.508075 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:11.510250 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:11.510267 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.512445 (+ 2178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.512453 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:11.512656 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:11.512693 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 13:00:11.512755 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 13:00:11.512880 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"thread_start_us":848,"threads_started":1}
I20260809 13:00:11.514613 8047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:11.503851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39610 (local address 127.21.235.67:44887)
0809 13:00:11.504875 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:11.504883 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:11.504912 (+ 29us) server_negotiation.cc:407] Connection header received
0809 13:00:11.505002 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:11.505010 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:11.505206 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:11.505407 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:00:11.506065 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.507905 (+ 1840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.510493 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.512200 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.513772 (+ 1572us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:11.514009 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:11.514083 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 13:00:11.514244 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":850,"thread_start_us":694,"threads_started":1}
I20260809 13:00:11.515908 7655 tablet_service.cc:1467] Processing CreateTablet for tablet a5750fe8f2354216b1f73c1dd72fc756 (DEFAULT_TABLE table=test-table [id=db19065ef3ea4cd28748176e441ff4d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:00:11.516171 7805 tablet_service.cc:1467] Processing CreateTablet for tablet a5750fe8f2354216b1f73c1dd72fc756 (DEFAULT_TABLE table=test-table [id=db19065ef3ea4cd28748176e441ff4d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:00:11.517390 7805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5750fe8f2354216b1f73c1dd72fc756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:00:11.517508 7655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5750fe8f2354216b1f73c1dd72fc756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:00:11.518167 7951 tablet_service.cc:1467] Processing CreateTablet for tablet a5750fe8f2354216b1f73c1dd72fc756 (DEFAULT_TABLE table=test-table [id=db19065ef3ea4cd28748176e441ff4d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:00:11.519635 7951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5750fe8f2354216b1f73c1dd72fc756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:00:11.529038 8048 tablet_bootstrap.cc:492] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef: Bootstrap starting.
I20260809 13:00:11.532302 8049 tablet_bootstrap.cc:492] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee: Bootstrap starting.
I20260809 13:00:11.534394 8048 tablet_bootstrap.cc:654] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef: Neither blocks nor log segments found. Creating new log.
I20260809 13:00:11.535136 8051 tablet_bootstrap.cc:492] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377: Bootstrap starting.
I20260809 13:00:11.535995 8049 tablet_bootstrap.cc:654] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee: Neither blocks nor log segments found. Creating new log.
I20260809 13:00:11.536293 8048 log.cc:824] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef: Log is configured to *not* fsync() on all Append() calls
I20260809 13:00:11.537307 8049 log.cc:824] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee: Log is configured to *not* fsync() on all Append() calls
I20260809 13:00:11.538516 8051 tablet_bootstrap.cc:654] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377: Neither blocks nor log segments found. Creating new log.
I20260809 13:00:11.539436 8051 log.cc:824] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377: Log is configured to *not* fsync() on all Append() calls
I20260809 13:00:11.539614 8048 tablet_bootstrap.cc:492] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef: No bootstrap required, opened a new log
I20260809 13:00:11.539919 8048 ts_tablet_manager.cc:1397] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260809 13:00:11.540495 8049 tablet_bootstrap.cc:492] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee: No bootstrap required, opened a new log
I20260809 13:00:11.540815 8049 ts_tablet_manager.cc:1397] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 13:00:11.542155 8051 tablet_bootstrap.cc:492] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377: No bootstrap required, opened a new log
I20260809 13:00:11.542446 8051 ts_tablet_manager.cc:1397] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260809 13:00:11.549568 8051 raft_consensus.cc:348] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } } peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } }
I20260809 13:00:11.549988 8051 raft_consensus.cc:374] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:00:11.550141 8051 raft_consensus.cc:729] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 390bb3a6078d436898a165c5dd8c8377, State: Initialized, Role: FOLLOWER
I20260809 13:00:11.550835 8051 consensus_queue.cc:260] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } } peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } }
I20260809 13:00:11.551796 8049 raft_consensus.cc:348] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } } peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } }
I20260809 13:00:11.552273 8049 raft_consensus.cc:374] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:00:11.552448 8049 raft_consensus.cc:729] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99bf84f74b948159d5dfebf961c10ee, State: Initialized, Role: FOLLOWER
I20260809 13:00:11.551779 8048 raft_consensus.cc:348] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } } peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } }
I20260809 13:00:11.552783 8048 raft_consensus.cc:374] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:00:11.552917 8048 raft_consensus.cc:729] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 929b08811b7e4e91b3cec796420a52ef, State: Initialized, Role: FOLLOWER
I20260809 13:00:11.553243 8049 consensus_queue.cc:260] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } } peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } }
I20260809 13:00:11.553483 8048 consensus_queue.cc:260] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } } peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } }
I20260809 13:00:11.555469 8049 ts_tablet_manager.cc:1428] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee: Time spent starting tablet: real 0.014s user 0.008s sys 0.005s
W20260809 13:00:11.556681 7730 tablet.cc:2369] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:00:11.557478 8051 ts_tablet_manager.cc:1428] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377: Time spent starting tablet: real 0.015s user 0.008s sys 0.006s
I20260809 13:00:11.557963 8026 heartbeater.cc:503] Master 127.21.235.126:42617 was elected leader, sending a full tablet report...
I20260809 13:00:11.561879 8048 ts_tablet_manager.cc:1428] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef: Time spent starting tablet: real 0.022s user 0.019s sys 0.000s
I20260809 13:00:11.562228 7875 heartbeater.cc:503] Master 127.21.235.126:42617 was elected leader, sending a full tablet report...
W20260809 13:00:11.604970 7876 tablet.cc:2369] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:00:11.676723 8027 tablet.cc:2369] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:00:11.802693 8058 raft_consensus.cc:482] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:00:11.803016 8058 raft_consensus.cc:504] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } } peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } }
I20260809 13:00:11.804768 8058 leader_election.cc:288] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99bf84f74b948159d5dfebf961c10ee (127.21.235.65:41273), 390bb3a6078d436898a165c5dd8c8377 (127.21.235.67:44887)
I20260809 13:00:11.812304 8043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:11.805310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:55431 (local address 127.21.235.65:41273)
0809 13:00:11.805550 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:11.805558 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:11.806386 (+ 828us) server_negotiation.cc:407] Connection header received
0809 13:00:11.806697 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:11.806701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:11.806761 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:11.806860 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:00:11.807542 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.808419 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.810457 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.811243 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.811455 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:11.811542 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:11.811894 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 13:00:11.812029 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 13:00:11.812296 8047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:11.805372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:44443 (local address 127.21.235.67:44887)
0809 13:00:11.805695 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:11.805703 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:11.806639 (+ 936us) server_negotiation.cc:407] Connection header received
0809 13:00:11.806853 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:11.806857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:11.806923 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:11.807023 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:00:11.807672 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.808432 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.810465 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.811242 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.811436 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:11.811529 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:11.811896 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 13:00:11.812033 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260809 13:00:11.812506 8059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:11.805267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:44887 (local address 127.21.235.66:44443)
0809 13:00:11.806521 (+ 1254us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:11.806536 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:00:11.806693 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:00:11.807042 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:00:11.807046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:00:11.807088 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:00:11.807505 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:11.807514 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.808568 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.808575 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:11.810269 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:11.810281 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.811509 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.811516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:11.811715 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:11.811760 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 13:00:11.811868 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 13:00:11.812000 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1103,"mutex_wait_us":105,"thread_start_us":481,"threads_started":1}
I20260809 13:00:11.812536 8060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:11.805231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:41273 (local address 127.21.235.66:55431)
0809 13:00:11.806240 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:11.806261 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:00:11.806530 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:00:11.806877 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:00:11.806881 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:00:11.806905 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:00:11.807372 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:11.807382 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.808560 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.808565 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:11.810270 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:11.810282 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.811545 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.811552 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:11.811719 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:11.811753 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 13:00:11.811857 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 13:00:11.811994 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":857,"mutex_wait_us":76,"spinlock_wait_cycles":21376,"thread_start_us":855,"threads_started":1}
I20260809 13:00:11.813609 7975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5750fe8f2354216b1f73c1dd72fc756" candidate_uuid: "929b08811b7e4e91b3cec796420a52ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "390bb3a6078d436898a165c5dd8c8377" is_pre_election: true
I20260809 13:00:11.813623 7680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5750fe8f2354216b1f73c1dd72fc756" candidate_uuid: "929b08811b7e4e91b3cec796420a52ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99bf84f74b948159d5dfebf961c10ee" is_pre_election: true
I20260809 13:00:11.814154 7680 raft_consensus.cc:2454] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 929b08811b7e4e91b3cec796420a52ef in term 0.
I20260809 13:00:11.814155 7975 raft_consensus.cc:2454] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 929b08811b7e4e91b3cec796420a52ef in term 0.
I20260809 13:00:11.814894 7753 leader_election.cc:302] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 390bb3a6078d436898a165c5dd8c8377, 929b08811b7e4e91b3cec796420a52ef; no voters:
I20260809 13:00:11.815294 8058 raft_consensus.cc:2781] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:00:11.815467 8058 raft_consensus.cc:482] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:00:11.815584 8058 raft_consensus.cc:3037] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:00:11.817838 8058 raft_consensus.cc:504] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } } peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } }
I20260809 13:00:11.818481 8058 leader_election.cc:288] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [CANDIDATE]: Term 1 election: Requested vote from peers f99bf84f74b948159d5dfebf961c10ee (127.21.235.65:41273), 390bb3a6078d436898a165c5dd8c8377 (127.21.235.67:44887)
I20260809 13:00:11.818953 7680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5750fe8f2354216b1f73c1dd72fc756" candidate_uuid: "929b08811b7e4e91b3cec796420a52ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99bf84f74b948159d5dfebf961c10ee"
I20260809 13:00:11.818965 7975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5750fe8f2354216b1f73c1dd72fc756" candidate_uuid: "929b08811b7e4e91b3cec796420a52ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "390bb3a6078d436898a165c5dd8c8377"
I20260809 13:00:11.819164 7680 raft_consensus.cc:3037] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:00:11.819173 7975 raft_consensus.cc:3037] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:00:11.821379 7975 raft_consensus.cc:2454] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 929b08811b7e4e91b3cec796420a52ef in term 1.
I20260809 13:00:11.821430 7680 raft_consensus.cc:2454] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 929b08811b7e4e91b3cec796420a52ef in term 1.
I20260809 13:00:11.821798 7753 leader_election.cc:302] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 390bb3a6078d436898a165c5dd8c8377, 929b08811b7e4e91b3cec796420a52ef; no voters:
I20260809 13:00:11.822080 8058 raft_consensus.cc:2781] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:00:11.822578 8058 raft_consensus.cc:686] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [term 1 LEADER]: Becoming Leader. State: Replica: 929b08811b7e4e91b3cec796420a52ef, State: Running, Role: LEADER
I20260809 13:00:11.823114 8058 consensus_queue.cc:237] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } } peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } }
I20260809 13:00:11.828059 7518 catalog_manager.cc:5495] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 929b08811b7e4e91b3cec796420a52ef (127.21.235.66). New cstate: current_term: 1 leader_uuid: "929b08811b7e4e91b3cec796420a52ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } health_report { overall_health: UNKNOWN } } }
I20260809 13:00:11.881134 8045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:11.875297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58294 (local address 127.21.235.66:46257)
0809 13:00:11.875550 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:11.875555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:11.875577 (+ 22us) server_negotiation.cc:407] Connection header received
0809 13:00:11.875812 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:11.875815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:11.875893 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:11.876040 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:00:11.876461 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.877194 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.878993 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.879394 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.879501 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:11.879994 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:00:11.880499 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:00:11.880617 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:11.880716 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 13:00:11.880784 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 13:00:11.890300 7975 raft_consensus.cc:1261] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377 [term 1 FOLLOWER]: Refusing update from remote peer 929b08811b7e4e91b3cec796420a52ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:00:11.890300 7680 raft_consensus.cc:1261] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee [term 1 FOLLOWER]: Refusing update from remote peer 929b08811b7e4e91b3cec796420a52ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:00:11.891551 8061 consensus_queue.cc:1035] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:00:11.892004 8058 consensus_queue.cc:1035] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:00:11.904825 8065 mvcc.cc:205] Tried to move back new op lower bound from 7316604567091265536 to 7316604566831599616. Current Snapshot: MvccSnapshot[applied={T|T < 7316604567091265536}]
I20260809 13:00:11.919478 7518 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:00:11.919734 7518 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:00:11.923840 7518 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: Sending DeleteTablet for 3 replicas of tablet a5750fe8f2354216b1f73c1dd72fc756
I20260809 13:00:11.925231 7951 tablet_service.cc:1514] Processing DeleteTablet for tablet a5750fe8f2354216b1f73c1dd72fc756 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:00:11 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:39610
I20260809 13:00:11.925256 7655 tablet_service.cc:1514] Processing DeleteTablet for tablet a5750fe8f2354216b1f73c1dd72fc756 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:00:11 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:40858
I20260809 13:00:11.925256 7805 tablet_service.cc:1514] Processing DeleteTablet for tablet a5750fe8f2354216b1f73c1dd72fc756 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:00:11 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:58284
I20260809 13:00:11.925660 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 7584
I20260809 13:00:11.946513 8079 tablet_replica.cc:330] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377: stopping tablet replica
I20260809 13:00:11.946928 8076 tablet_replica.cc:330] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef: stopping tablet replica
I20260809 13:00:11.947239 8079 raft_consensus.cc:2229] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:00:11.947854 8079 raft_consensus.cc:2258] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:00:11.947809 8076 raft_consensus.cc:2229] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:00:11.948688 8078 tablet_replica.cc:330] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee: stopping tablet replica
I20260809 13:00:11.949169 8078 raft_consensus.cc:2229] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:00:11.949832 8078 raft_consensus.cc:2258] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:00:11.951269 8079 ts_tablet_manager.cc:1905] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:00:11.950178 8076 raft_consensus.cc:2258] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 09 13:00:11 dist-test-slave-gp6n krb5kdc[7467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 13:00:11.952131 8078 ts_tablet_manager.cc:1905] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:00:11.954062 8076 ts_tablet_manager.cc:1905] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:00:11.960526 8078 ts_tablet_manager.cc:1918] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:00:11.960806 8078 log.cc:1196] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/a5750fe8f2354216b1f73c1dd72fc756
I20260809 13:00:11.961315 8079 ts_tablet_manager.cc:1918] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:00:11.961469 8078 ts_tablet_manager.cc:1939] T a5750fe8f2354216b1f73c1dd72fc756 P f99bf84f74b948159d5dfebf961c10ee: Deleting consensus metadata
I20260809 13:00:11.961807 8043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:11.927185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40864 (local address 127.21.235.65:41273)
0809 13:00:11.927428 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:11.927438 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:11.937198 (+ 9760us) server_negotiation.cc:407] Connection header received
0809 13:00:11.937750 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:11.937757 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:11.937857 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:11.937948 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:11.940154 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.941451 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.943886 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:11.944852 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:11.944981 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:11.953227 (+ 8246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:11.953247 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:11.953264 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:11.953311 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:11.957448 (+ 4137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:11.958566 (+ 1118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:11.958580 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:11.958584 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:11.958761 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:11.959239 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:11.959245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:11.959249 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:11.959876 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:11.960183 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:11.960839 (+ 656us) server_negotiation.cc:299] Negotiation successful
0809 13:00:11.961222 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 13:00:11.963613 7723 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:00:11.963797 8079 log.cc:1196] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/a5750fe8f2354216b1f73c1dd72fc756
I20260809 13:00:11.964182 7503 catalog_manager.cc:4841] TS f99bf84f74b948159d5dfebf961c10ee (127.21.235.65:41273): tablet a5750fe8f2354216b1f73c1dd72fc756 (table test-table [id=db19065ef3ea4cd28748176e441ff4d7]) successfully deleted
I20260809 13:00:11.964452 8079 ts_tablet_manager.cc:1939] T a5750fe8f2354216b1f73c1dd72fc756 P 390bb3a6078d436898a165c5dd8c8377: Deleting consensus metadata
I20260809 13:00:11.966598 7502 catalog_manager.cc:4841] TS 390bb3a6078d436898a165c5dd8c8377 (127.21.235.67:44887): tablet a5750fe8f2354216b1f73c1dd72fc756 (table test-table [id=db19065ef3ea4cd28748176e441ff4d7]) successfully deleted
I20260809 13:00:11.971462 8076 ts_tablet_manager.cc:1918] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:00:11.971771 8076 log.cc:1196] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/a5750fe8f2354216b1f73c1dd72fc756
I20260809 13:00:11.972702 8076 ts_tablet_manager.cc:1939] T a5750fe8f2354216b1f73c1dd72fc756 P 929b08811b7e4e91b3cec796420a52ef: Deleting consensus metadata
I20260809 13:00:11.974637 7502 catalog_manager.cc:4841] TS 929b08811b7e4e91b3cec796420a52ef (127.21.235.66:46257): tablet a5750fe8f2354216b1f73c1dd72fc756 (table test-table [id=db19065ef3ea4cd28748176e441ff4d7]) successfully deleted
I20260809 13:00:12.079478 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 7584
I20260809 13:00:12.104161 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 7736
Aug 09 13:00:12 dist-test-slave-gp6n krb5kdc[7467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.66@KRBTEST.COM
I20260809 13:00:12.124712 8045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:12.105102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58306 (local address 127.21.235.66:46257)
0809 13:00:12.105319 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:12.105326 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:12.105403 (+ 77us) server_negotiation.cc:407] Connection header received
0809 13:00:12.105675 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:12.105681 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:12.105750 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:12.105849 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:12.107771 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:12.108723 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:12.110936 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:12.111437 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:12.111585 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:12.117241 (+ 5656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:12.117264 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:12.117285 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:12.117335 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:12.121508 (+ 4173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:12.122370 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:12.122380 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:12.122385 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:12.122596 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:12.123015 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:12.123020 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:12.123023 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:12.123493 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:12.123856 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:12.124059 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 13:00:12.124301 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 13:00:12.125662 7868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:00:12.248647 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 7736
I20260809 13:00:12.270601 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 7882
Aug 09 13:00:12 dist-test-slave-gp6n krb5kdc[7467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.67@KRBTEST.COM
I20260809 13:00:12.285698 8047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:12.271384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39616 (local address 127.21.235.67:44887)
0809 13:00:12.271601 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:12.271606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:12.271669 (+ 63us) server_negotiation.cc:407] Connection header received
0809 13:00:12.271816 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:12.271820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:12.271884 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:12.271989 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:12.273274 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:12.273984 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:12.275377 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:12.275755 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:12.275848 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:12.280272 (+ 4424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:12.280288 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:12.280304 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:12.280335 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:12.283053 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:12.283843 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:12.283851 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:12.283854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:12.284004 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:12.284328 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:12.284331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:12.284333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:12.284752 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:12.284984 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:12.285195 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 13:00:12.285364 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 13:00:12.286725 8021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:00:12.376626 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 7882
I20260809 13:00:12.395503 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 7484
I20260809 13:00:12.396709 7549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:00:12.526217 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 7484
I20260809 13:00:32.552338 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:42617
--webserver_interface=127.21.235.126
--webserver_port=40127
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:46303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:42617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:00:32.920282 8085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:00:32.920637 8085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:00:32.920804 8085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:00:32.928076 8085 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 13:00:32.928221 8085 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 13:00:32.928326 8085 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 13:00:32.928437 8085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:00:32.928543 8085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:00:32.928666 8085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:00:32.928754 8085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:00:32.928834 8085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:00:32.938915 8085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:46303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:42617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:42617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=40127
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:00:32.940305 8085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:00:32.941747 8085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:00:32.967731 8094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:32.967736 8098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:32.967819 8096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:00:32.972047 8085 server_base.cc:1034] running on GCE node
I20260809 13:00:32.973163 8085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:00:32.974876 8085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:00:32.976138 8085 hybrid_clock.cc:648] HybridClock initialized: now 1786280432976031 us; error 56 us; skew 500 ppm
I20260809 13:00:32.976624 8085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:00:32 dist-test-slave-gp6n krb5kdc[7467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280432, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:00:32.980567 8085 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 13:00:32.982313 8085 webserver.cc:460] Webserver started at http://127.21.235.126:40127/ using document root <none> and password file <none>
I20260809 13:00:32.983156 8085 fs_manager.cc:359] Metadata directory not provided
I20260809 13:00:32.983297 8085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:00:32.989427 8085 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 13:00:32.992656 8105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:32.994122 8085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:00:32.994321 8085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "6b7e946c4e184052afd2f872047d8e42"
format_stamp: "Formatted at 2026-08-09 13:00:09 on dist-test-slave-gp6n"
server_key: "12c95c171f36288dc31c1a6cc4842007"
server_key_iv: "2ec67b21f6cf591f52199be3f9c4f3ce"
server_key_version: "encryptionkey@0"
I20260809 13:00:32.995450 8085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:00:33.017449 8085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:00:33.022149 8085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:00:33.022548 8085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:00:33.042212 8085 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:42617
I20260809 13:00:33.042236 8161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:42617 every 8 connection(s)
I20260809 13:00:33.043851 8085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:00:33.049649 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 8085
I20260809 13:00:33.050794 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:41273
--local_ip_for_outbound_sockets=127.21.235.65
--tserver_master_addrs=127.21.235.126:42617
--webserver_port=45979
--webserver_interface=127.21.235.65
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:46303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:00:33.052003 8162 sys_catalog.cc:263] Verifying existing consensus state
I20260809 13:00:33.057363 8162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: Bootstrap starting.
I20260809 13:00:33.079578 8162 log.cc:824] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: Log is configured to *not* fsync() on all Append() calls
I20260809 13:00:33.092056 8162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 13:00:33.092640 8162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: Bootstrap complete.
I20260809 13:00:33.102623 8162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7e946c4e184052afd2f872047d8e42" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42617 } }
I20260809 13:00:33.103505 8162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b7e946c4e184052afd2f872047d8e42, State: Initialized, Role: FOLLOWER
I20260809 13:00:33.104187 8162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7e946c4e184052afd2f872047d8e42" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42617 } }
I20260809 13:00:33.104446 8162 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:00:33.104620 8162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:00:33.104808 8162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 1 FOLLOWER]: Advancing to term 2
I20260809 13:00:33.108744 8162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7e946c4e184052afd2f872047d8e42" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42617 } }
I20260809 13:00:33.109259 8162 leader_election.cc:302] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b7e946c4e184052afd2f872047d8e42; no voters:
I20260809 13:00:33.109920 8162 leader_election.cc:288] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 13:00:33.110288 8166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 13:00:33.111856 8166 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 2 LEADER]: Becoming Leader. State: Replica: 6b7e946c4e184052afd2f872047d8e42, State: Running, Role: LEADER
I20260809 13:00:33.112612 8162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:00:33.112578 8166 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7e946c4e184052afd2f872047d8e42" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42617 } }
I20260809 13:00:33.120627 8168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b7e946c4e184052afd2f872047d8e42. Latest consensus state: current_term: 2 leader_uuid: "6b7e946c4e184052afd2f872047d8e42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7e946c4e184052afd2f872047d8e42" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42617 } } }
I20260809 13:00:33.120981 8168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [sys.catalog]: This master's current role is: LEADER
I20260809 13:00:33.116726 8167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6b7e946c4e184052afd2f872047d8e42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7e946c4e184052afd2f872047d8e42" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42617 } } }
I20260809 13:00:33.121510 8167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [sys.catalog]: This master's current role is: LEADER
I20260809 13:00:33.127593 8177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:00:33.135926 8177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:00:33.137538 8177 catalog_manager.cc:1259] Loaded cluster ID: a31abf667e9f4c52aebd5855c2b155f8
I20260809 13:00:33.137679 8177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:00:33.141806 8177 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:00:33.143539 8177 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: Loaded TSK: 0
I20260809 13:00:33.175174 8177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: Generated new TSK 1
I20260809 13:00:33.175642 8177 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 13:00:33.418220 8164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:00:33.418605 8164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:00:33.418771 8164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:00:33.426033 8164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:00:33.426185 8164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:00:33.426384 8164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 13:00:33.436251 8164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:46303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:41273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=45979
--tserver_master_addrs=127.21.235.126:42617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:00:33.437613 8164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:00:33.439081 8164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:00:33.454424 8195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:33.454890 8193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:33.454960 8197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:00:33.455862 8164 server_base.cc:1034] running on GCE node
I20260809 13:00:33.456681 8164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:00:33.457974 8164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:00:33.459220 8164 hybrid_clock.cc:648] HybridClock initialized: now 1786280433459173 us; error 31 us; skew 500 ppm
I20260809 13:00:33.459613 8164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:00:33 dist-test-slave-gp6n krb5kdc[7467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280433, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:00:33.468806 8164 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 13:00:33.470903 8164 webserver.cc:460] Webserver started at http://127.21.235.65:45979/ using document root <none> and password file <none>
I20260809 13:00:33.471931 8164 fs_manager.cc:359] Metadata directory not provided
I20260809 13:00:33.472134 8164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:00:33.480206 8164 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 13:00:33.483965 8205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:33.485630 8164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:00:33.485821 8164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "f99bf84f74b948159d5dfebf961c10ee"
format_stamp: "Formatted at 2026-08-09 13:00:10 on dist-test-slave-gp6n"
server_key: "c6727ef2bc6b1b70e3a1d3451fe8116f"
server_key_iv: "d84bf348ff324e47609313cef9a57d41"
server_key_version: "encryptionkey@0"
I20260809 13:00:33.487322 8164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:00:33.509827 8164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:00:33.514690 8164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:00:33.514940 8164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:00:33.515928 8164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:00:33.517222 8164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:00:33.517319 8164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:33.517398 8164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:00:33.517458 8164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:33.553339 8164 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:41273
I20260809 13:00:33.553467 8331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:41273 every 8 connection(s)
I20260809 13:00:33.554870 8164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 13:00:33.563284 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 8164
I20260809 13:00:33.564257 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:46257
--local_ip_for_outbound_sockets=127.21.235.66
--tserver_master_addrs=127.21.235.126:42617
--webserver_port=37971
--webserver_interface=127.21.235.66
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:46303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 13:00:33 dist-test-slave-gp6n krb5kdc[7467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280433, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:00:33.583592 8336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:33.558056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:44381)
0809 13:00:33.558963 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:33.559019 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 13:00:33.560800 (+ 1781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:00:33.562126 (+ 1326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:00:33.562133 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:00:33.562867 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:00:33.563402 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:33.563420 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:33.566183 (+ 2763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:33.566187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:33.567710 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:33.567725 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:33.569438 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:33.569448 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:33.569771 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:33.570694 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:00:33.570729 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:00:33.574200 (+ 3471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:00:33.578534 (+ 4334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:00:33.578544 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:00:33.578562 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:00:33.579063 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:00:33.579606 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:00:33.579611 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:00:33.579616 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:00:33.579843 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:00:33.580701 (+ 858us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:00:33.580710 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:00:33.581103 (+ 393us) client_negotiation.cc:769] Sending connection context
0809 13:00:33.581422 (+ 319us) client_negotiation.cc:240] Negotiation successful
0809 13:00:33.582061 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":378,"threads_started":1}
I20260809 13:00:33.583592 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:33.557991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:44381 (local address 127.21.235.126:42617)
0809 13:00:33.558777 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:33.558790 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:33.559150 (+ 360us) server_negotiation.cc:407] Connection header received
0809 13:00:33.561352 (+ 2202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:33.561366 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:33.561685 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:33.561986 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:33.563649 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:33.565951 (+ 2302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:33.567892 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:33.569132 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:33.569370 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:33.574602 (+ 5232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:33.574616 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:33.574641 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:33.574697 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:33.578317 (+ 3620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:33.579246 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:33.579254 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:33.579260 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:33.579459 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:33.579993 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:33.579997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:33.580000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:33.580537 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:33.580783 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:33.581693 (+ 910us) server_negotiation.cc:299] Negotiation successful
0809 13:00:33.582121 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":377,"threads_started":1}
I20260809 13:00:33.586766 8332 heartbeater.cc:347] Connected to a master server at 127.21.235.126:42617
I20260809 13:00:33.587179 8332 heartbeater.cc:464] Registering TS with master...
I20260809 13:00:33.588171 8332 heartbeater.cc:511] Master 127.21.235.126:42617 requested a full tablet report, sending...
I20260809 13:00:33.590842 8124 ts_manager.cc:194] Registered new tserver with Master: f99bf84f74b948159d5dfebf961c10ee (127.21.235.65:41273)
I20260809 13:00:33.592417 8124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:44381
W20260809 13:00:33.924279 8337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:00:33.924599 8337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:00:33.924732 8337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:00:33.931866 8337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:00:33.931977 8337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:00:33.932096 8337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 13:00:33.941669 8337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:46303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:46257
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=37971
--tserver_master_addrs=127.21.235.126:42617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:00:33.942991 8337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:00:33.944283 8337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:00:33.957504 8342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:33.957579 8343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:33.957583 8346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:00:33.958902 8337 server_base.cc:1034] running on GCE node
I20260809 13:00:33.959667 8337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:00:33.960677 8337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:00:33.961959 8337 hybrid_clock.cc:648] HybridClock initialized: now 1786280433961909 us; error 39 us; skew 500 ppm
I20260809 13:00:33.962296 8337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:00:33 dist-test-slave-gp6n krb5kdc[7467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280433, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:00:33.965672 8337 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 13:00:33.967136 8337 webserver.cc:460] Webserver started at http://127.21.235.66:37971/ using document root <none> and password file <none>
I20260809 13:00:33.967767 8337 fs_manager.cc:359] Metadata directory not provided
I20260809 13:00:33.967887 8337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:00:33.973075 8337 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 13:00:33.975740 8356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:33.977031 8337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:00:33.977200 8337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "929b08811b7e4e91b3cec796420a52ef"
format_stamp: "Formatted at 2026-08-09 13:00:10 on dist-test-slave-gp6n"
server_key: "b6ba40033a26cf8ec5130ffb473161ab"
server_key_iv: "dfedc33a3e3196d88210cc82350c8c7c"
server_key_version: "encryptionkey@0"
I20260809 13:00:33.978139 8337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:00:33.996670 8337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:00:34.001268 8337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:00:34.001555 8337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:00:34.002825 8337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:00:34.004160 8337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:00:34.004280 8337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:34.004415 8337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:00:34.004523 8337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:34.041107 8337 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:46257
I20260809 13:00:34.041199 8479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:46257 every 8 connection(s)
I20260809 13:00:34.042667 8337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 13:00:34.047165 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 8337
I20260809 13:00:34.047924 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:44887
--local_ip_for_outbound_sockets=127.21.235.67
--tserver_master_addrs=127.21.235.126:42617
--webserver_port=37681
--webserver_interface=127.21.235.67
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:46303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 13:00:34 dist-test-slave-gp6n krb5kdc[7467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280433, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:00:34.064693 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:34.046526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:59495 (local address 127.21.235.126:42617)
0809 13:00:34.046730 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:34.046736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:34.046923 (+ 187us) server_negotiation.cc:407] Connection header received
0809 13:00:34.049155 (+ 2232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:34.049161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:34.049290 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:34.049403 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:34.050990 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.051697 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.053251 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.053734 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.053882 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:34.057741 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:34.057746 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:34.057750 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:34.057811 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:34.061204 (+ 3393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:34.061951 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:34.061956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:34.061962 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:34.062096 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:34.062664 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:34.062672 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:34.062678 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:34.063067 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:34.063216 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:34.064109 (+ 893us) server_negotiation.cc:299] Negotiation successful
0809 13:00:34.064340 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 13:00:34.066236 8482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:34.045901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:59495)
0809 13:00:34.046753 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:34.046798 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 13:00:34.048881 (+ 2083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:00:34.049660 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:00:34.049668 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:00:34.050172 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:00:34.050819 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:34.050838 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.051869 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.051874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:34.053106 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:34.053118 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.053960 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.053967 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:34.054305 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:34.055233 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:00:34.055255 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:00:34.057497 (+ 2242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:00:34.061388 (+ 3891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:00:34.061396 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:00:34.061411 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:00:34.061818 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:00:34.062208 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:00:34.062217 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:00:34.062222 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:00:34.062504 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:00:34.063209 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:00:34.063219 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:00:34.063681 (+ 462us) client_negotiation.cc:769] Sending connection context
0809 13:00:34.063986 (+ 305us) client_negotiation.cc:240] Negotiation successful
0809 13:00:34.064555 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":349,"threads_started":1}
I20260809 13:00:34.067802 8480 heartbeater.cc:347] Connected to a master server at 127.21.235.126:42617
I20260809 13:00:34.068151 8480 heartbeater.cc:464] Registering TS with master...
I20260809 13:00:34.068941 8480 heartbeater.cc:511] Master 127.21.235.126:42617 requested a full tablet report, sending...
I20260809 13:00:34.070468 8124 ts_manager.cc:194] Registered new tserver with Master: 929b08811b7e4e91b3cec796420a52ef (127.21.235.66:46257)
I20260809 13:00:34.071283 8124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:59495
W20260809 13:00:34.405190 8483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:00:34.405459 8483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:00:34.405553 8483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:00:34.412848 8483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:00:34.412985 8483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:00:34.413137 8483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 13:00:34.422816 8483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:46303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:44887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=37681
--tserver_master_addrs=127.21.235.126:42617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:00:34.424072 8483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:00:34.425371 8483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:00:34.438925 8488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:34.438908 8491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:34.438966 8489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:00:34.440688 8483 server_base.cc:1034] running on GCE node
I20260809 13:00:34.441336 8483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:00:34.442302 8483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:00:34.443539 8483 hybrid_clock.cc:648] HybridClock initialized: now 1786280434443495 us; error 34 us; skew 500 ppm
I20260809 13:00:34.443867 8483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:00:34 dist-test-slave-gp6n krb5kdc[7467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280434, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:00:34.447199 8483 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 13:00:34.448874 8483 webserver.cc:460] Webserver started at http://127.21.235.67:37681/ using document root <none> and password file <none>
I20260809 13:00:34.449548 8483 fs_manager.cc:359] Metadata directory not provided
I20260809 13:00:34.449695 8483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:00:34.455211 8483 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 13:00:34.458006 8498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:34.459626 8483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:00:34.459769 8483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "390bb3a6078d436898a165c5dd8c8377"
format_stamp: "Formatted at 2026-08-09 13:00:11 on dist-test-slave-gp6n"
server_key: "6c662aa6a0db5f02d02b1fc92309573d"
server_key_iv: "766ef75f8bd3ad689e67ae4e325729c3"
server_key_version: "encryptionkey@0"
I20260809 13:00:34.460714 8483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:00:34.482752 8483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:00:34.487066 8483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:00:34.487282 8483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:00:34.488170 8483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:00:34.489516 8483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:00:34.489610 8483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:34.489694 8483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:00:34.489748 8483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:34.525200 8483 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:44887
I20260809 13:00:34.525292 8626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:44887 every 8 connection(s)
I20260809 13:00:34.526695 8483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 13:00:34.532769 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 8483
Aug 09 13:00:34 dist-test-slave-gp6n krb5kdc[7467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280434, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:00:34.549098 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:34.529425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:39681 (local address 127.21.235.126:42617)
0809 13:00:34.529612 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:34.529617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:34.530806 (+ 1189us) server_negotiation.cc:407] Connection header received
0809 13:00:34.532960 (+ 2154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:34.532973 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:34.533045 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:34.533135 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:34.535130 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.535839 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.537738 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.538177 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.538267 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:34.542339 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:34.542347 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:34.542352 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:34.542395 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:34.545432 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:34.546355 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:34.546359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:34.546361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:34.546556 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:34.547073 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:34.547078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:34.547081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:34.547440 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:34.547599 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:34.548571 (+ 972us) server_negotiation.cc:299] Negotiation successful
0809 13:00:34.548734 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 13:00:34.550218 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:34.533904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33900 (local address 127.21.235.126:42617)
0809 13:00:34.534730 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:34.534735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:34.534750 (+ 15us) server_negotiation.cc:407] Connection header received
0809 13:00:34.534792 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:34.534795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:34.534854 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:34.534937 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:34.536878 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.537611 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.539567 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.539988 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.540076 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:34.543424 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:34.543429 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:34.543432 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:34.543474 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:34.546914 (+ 3440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:34.547784 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:34.547790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:34.547794 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:34.547949 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:34.548401 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:34.548408 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:34.548414 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:34.548809 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:34.549139 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:34.549530 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 13:00:34.549773 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":625,"threads_started":1}
I20260809 13:00:34.550848 8629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:34.529870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:39681)
0809 13:00:34.530644 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:34.530683 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 13:00:34.532601 (+ 1918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:00:34.533454 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:00:34.533469 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:00:34.534077 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:00:34.534953 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:34.534980 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.536012 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.536020 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:34.537589 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:34.537601 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.538444 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.538454 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:34.538735 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:34.539448 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:00:34.539491 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:00:34.542118 (+ 2627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:00:34.545582 (+ 3464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:00:34.545593 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:00:34.545612 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:00:34.546226 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:00:34.546673 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:00:34.546680 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:00:34.546685 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:00:34.546949 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:00:34.547600 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:00:34.547607 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:00:34.548120 (+ 513us) client_negotiation.cc:769] Sending connection context
0809 13:00:34.548461 (+ 341us) client_negotiation.cc:240] Negotiation successful
0809 13:00:34.549095 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":279,"threads_started":1}
I20260809 13:00:34.552393 8627 heartbeater.cc:347] Connected to a master server at 127.21.235.126:42617
I20260809 13:00:34.552747 8627 heartbeater.cc:464] Registering TS with master...
I20260809 13:00:34.553696 8627 heartbeater.cc:511] Master 127.21.235.126:42617 requested a full tablet report, sending...
I20260809 13:00:34.555359 8124 ts_manager.cc:194] Registered new tserver with Master: 390bb3a6078d436898a165c5dd8c8377 (127.21.235.67:44887)
I20260809 13:00:34.556171 8124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:39681
I20260809 13:00:34.565563 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:00:34.572508 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:34.566664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33912 (local address 127.21.235.126:42617)
0809 13:00:34.566896 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:34.566905 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:34.567426 (+ 521us) server_negotiation.cc:407] Connection header received
0809 13:00:34.567606 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:34.567611 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:34.567740 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:34.567860 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:00:34.568330 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.569083 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.570763 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.571168 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.571269 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:34.571782 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:00:34.572036 (+ 254us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 13:00:34.572315 (+ 279us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33912: token expired
Metrics: {"server-negotiator.queue_time_us":66}
W20260809 13:00:34.572640 8631 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33912: token expired
W20260809 13:00:34.573251 8033 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.21.235.126:42617)
I20260809 13:00:34.586536 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:34.573793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33918 (local address 127.21.235.126:42617)
0809 13:00:34.573958 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:34.573965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:34.574041 (+ 76us) server_negotiation.cc:407] Connection header received
0809 13:00:34.574216 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:34.574221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:34.574281 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:34.574388 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:34.575821 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.576525 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.577882 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.578281 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.578457 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:34.581019 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:34.581024 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:34.581027 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:34.581057 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:34.583767 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:34.584499 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:34.584511 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:34.584513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:34.584640 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:34.585030 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:34.585036 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:34.585041 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:34.585341 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:34.585616 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:34.585926 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 13:00:34.586149 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 13:00:34.589969 8124 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 13:00:34.596782 8332 heartbeater.cc:503] Master 127.21.235.126:42617 was elected leader, sending a full tablet report...
W20260809 13:00:34.597455 8124 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:00:34.622709 8639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:34.610421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:41273 (local address 127.0.0.1:49350)
0809 13:00:34.612807 (+ 2386us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:34.612832 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 13:00:34.612986 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:00:34.614230 (+ 1244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:00:34.614242 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:00:34.614285 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:00:34.614805 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:34.614827 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.617457 (+ 2630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.617464 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:34.619475 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:34.619490 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.621576 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.621588 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:34.621783 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:34.621838 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 13:00:34.621992 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 13:00:34.622173 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2229,"mutex_wait_us":37,"thread_start_us":1729,"threads_started":1}
I20260809 13:00:34.622875 8638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:34.610008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42346 (local address 127.21.235.67:44887)
0809 13:00:34.614808 (+ 4800us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:34.614817 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:34.614848 (+ 31us) server_negotiation.cc:407] Connection header received
0809 13:00:34.614944 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:34.614953 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:34.615214 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:34.616122 (+ 908us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:00:34.616430 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.618023 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.620559 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.621757 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.621912 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:34.622243 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:34.622316 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 13:00:34.622502 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4626,"thread_start_us":4459,"threads_started":1}
I20260809 13:00:34.622958 8637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:34.609620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:44887 (local address 127.0.0.1:42346)
0809 13:00:34.611063 (+ 1443us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:34.611111 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 13:00:34.611317 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:00:34.615465 (+ 4148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:00:34.615469 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:00:34.615500 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:00:34.615933 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:34.615945 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.618165 (+ 2220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.618170 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:34.620371 (+ 2201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:34.620386 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.621975 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.621982 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:34.622148 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:34.622184 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 13:00:34.622274 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 13:00:34.622448 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1176,"thread_start_us":1050,"threads_started":1}
I20260809 13:00:34.623541 8642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:34.611846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46718 (local address 127.21.235.66:46257)
0809 13:00:34.613926 (+ 2080us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:34.613937 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:34.614717 (+ 780us) server_negotiation.cc:407] Connection header received
0809 13:00:34.614835 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:34.614844 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:34.615081 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:34.615278 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:00:34.615840 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.617181 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.620090 (+ 2909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.621894 (+ 1804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.622017 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:34.622186 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:34.623061 (+ 875us) server_negotiation.cc:299] Negotiation successful
0809 13:00:34.623225 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1876,"thread_start_us":668,"threads_started":1}
I20260809 13:00:34.623967 8641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:34.611595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:46257 (local address 127.0.0.1:46718)
0809 13:00:34.614492 (+ 2897us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:34.614523 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 13:00:34.614660 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:00:34.615260 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:00:34.615265 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:00:34.615291 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:00:34.615682 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:34.615693 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.617327 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.617334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:34.619919 (+ 2585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:34.619936 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.622364 (+ 2428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.622370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:34.622597 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:34.622644 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 13:00:34.622746 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 13:00:34.622905 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2718,"mutex_wait_us":85,"thread_start_us":2521,"threads_started":1}
I20260809 13:00:34.624364 8640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:34.610769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49350 (local address 127.21.235.65:41273)
0809 13:00:34.611967 (+ 1198us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:34.611976 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:34.613148 (+ 1172us) server_negotiation.cc:407] Connection header received
0809 13:00:34.613277 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:34.613288 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:34.613617 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:34.613847 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:00:34.615011 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.617275 (+ 2264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.619641 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.621030 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.621172 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:34.621428 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:34.623197 (+ 1769us) server_negotiation.cc:299] Negotiation successful
0809 13:00:34.623383 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1032,"thread_start_us":1288,"threads_started":1}
I20260809 13:00:34.627513 8259 tablet_service.cc:1467] Processing CreateTablet for tablet e1df4621f85c4315abbf260eff069e6e (DEFAULT_TABLE table=test-table [id=127de2d121fb42398ed73676003997a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:00:34.627478 8552 tablet_service.cc:1467] Processing CreateTablet for tablet e1df4621f85c4315abbf260eff069e6e (DEFAULT_TABLE table=test-table [id=127de2d121fb42398ed73676003997a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:00:34.628809 8259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1df4621f85c4315abbf260eff069e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:00:34.628930 8552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1df4621f85c4315abbf260eff069e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:00:34.628930 8410 tablet_service.cc:1467] Processing CreateTablet for tablet e1df4621f85c4315abbf260eff069e6e (DEFAULT_TABLE table=test-table [id=127de2d121fb42398ed73676003997a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:00:34.630180 8410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1df4621f85c4315abbf260eff069e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:00:34.640712 8643 tablet_bootstrap.cc:492] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee: Bootstrap starting.
I20260809 13:00:34.643950 8643 tablet_bootstrap.cc:654] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee: Neither blocks nor log segments found. Creating new log.
I20260809 13:00:34.644997 8643 log.cc:824] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee: Log is configured to *not* fsync() on all Append() calls
I20260809 13:00:34.646127 8644 tablet_bootstrap.cc:492] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377: Bootstrap starting.
I20260809 13:00:34.648662 8645 tablet_bootstrap.cc:492] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef: Bootstrap starting.
I20260809 13:00:34.650029 8643 tablet_bootstrap.cc:492] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee: No bootstrap required, opened a new log
I20260809 13:00:34.650331 8643 ts_tablet_manager.cc:1397] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 13:00:34.651023 8644 tablet_bootstrap.cc:654] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377: Neither blocks nor log segments found. Creating new log.
I20260809 13:00:34.652277 8644 log.cc:824] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377: Log is configured to *not* fsync() on all Append() calls
I20260809 13:00:34.652611 8645 tablet_bootstrap.cc:654] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef: Neither blocks nor log segments found. Creating new log.
I20260809 13:00:34.653833 8645 log.cc:824] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef: Log is configured to *not* fsync() on all Append() calls
I20260809 13:00:34.655504 8644 tablet_bootstrap.cc:492] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377: No bootstrap required, opened a new log
I20260809 13:00:34.655779 8644 ts_tablet_manager.cc:1397] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 13:00:34.656879 8645 tablet_bootstrap.cc:492] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef: No bootstrap required, opened a new log
I20260809 13:00:34.657137 8645 ts_tablet_manager.cc:1397] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 13:00:34.658862 8643 raft_consensus.cc:348] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } } peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } }
I20260809 13:00:34.659353 8643 raft_consensus.cc:374] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:00:34.659525 8643 raft_consensus.cc:729] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99bf84f74b948159d5dfebf961c10ee, State: Initialized, Role: FOLLOWER
I20260809 13:00:34.660133 8643 consensus_queue.cc:260] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } } peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } }
I20260809 13:00:34.661926 8643 ts_tablet_manager.cc:1428] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260809 13:00:34.666766 8644 raft_consensus.cc:348] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } } peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } }
I20260809 13:00:34.667191 8644 raft_consensus.cc:374] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:00:34.667289 8644 raft_consensus.cc:729] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 390bb3a6078d436898a165c5dd8c8377, State: Initialized, Role: FOLLOWER
I20260809 13:00:34.667940 8644 consensus_queue.cc:260] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } } peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } }
I20260809 13:00:34.667836 8645 raft_consensus.cc:348] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } } peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } }
I20260809 13:00:34.668254 8645 raft_consensus.cc:374] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:00:34.668370 8645 raft_consensus.cc:729] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 929b08811b7e4e91b3cec796420a52ef, State: Initialized, Role: FOLLOWER
I20260809 13:00:34.669057 8627 heartbeater.cc:503] Master 127.21.235.126:42617 was elected leader, sending a full tablet report...
I20260809 13:00:34.668983 8645 consensus_queue.cc:260] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } } peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } }
I20260809 13:00:34.669804 8644 ts_tablet_manager.cc:1428] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377: Time spent starting tablet: real 0.014s user 0.007s sys 0.007s
I20260809 13:00:34.670126 8480 heartbeater.cc:503] Master 127.21.235.126:42617 was elected leader, sending a full tablet report...
I20260809 13:00:34.670657 8645 ts_tablet_manager.cc:1428] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef: Time spent starting tablet: real 0.013s user 0.010s sys 0.005s
W20260809 13:00:34.778369 8628 tablet.cc:2369] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:00:34.794689 8481 tablet.cc:2369] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:00:34.807334 8333 tablet.cc:2369] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:00:34.951331 8649 raft_consensus.cc:482] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:00:34.951717 8649 raft_consensus.cc:504] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } } peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } }
I20260809 13:00:34.953586 8649 leader_election.cc:288] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 390bb3a6078d436898a165c5dd8c8377 (127.21.235.67:44887), 929b08811b7e4e91b3cec796420a52ef (127.21.235.66:46257)
I20260809 13:00:34.960717 8638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:34.954102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:40017 (local address 127.21.235.67:44887)
0809 13:00:34.954295 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:34.954300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:34.954694 (+ 394us) server_negotiation.cc:407] Connection header received
0809 13:00:34.955054 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:34.955063 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:34.955144 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:34.955212 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:00:34.955937 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.956660 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.958803 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.959689 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.959877 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:34.959953 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:34.960290 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 13:00:34.960446 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 13:00:34.960876 8642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:34.954978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:38527 (local address 127.21.235.66:46257)
0809 13:00:34.955202 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:34.955210 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:34.955465 (+ 255us) server_negotiation.cc:407] Connection header received
0809 13:00:34.955686 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:34.955691 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:34.955756 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:34.955857 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:00:34.956426 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.957100 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.959026 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.959881 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.960069 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:34.960143 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:34.960476 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 13:00:34.960622 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 13:00:34.961124 8652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:34.953858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:44887 (local address 127.21.235.65:40017)
0809 13:00:34.954587 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:34.954609 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 13:00:34.954832 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:00:34.955417 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:00:34.955420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:00:34.955442 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:00:34.955794 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:34.955802 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.956799 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.956804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:34.958656 (+ 1852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:34.958669 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.959892 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.959898 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:34.960061 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:34.960089 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 13:00:34.960227 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 13:00:34.960396 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":61,"spinlock_wait_cycles":27520,"thread_start_us":350,"threads_started":1}
I20260809 13:00:34.961198 8653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:34.954739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:46257 (local address 127.21.235.65:38527)
0809 13:00:34.955358 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:34.955374 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:00:34.955534 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:00:34.955877 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:00:34.955881 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:00:34.955905 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:00:34.956236 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:34.956245 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.957234 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.957241 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:34.958869 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:00:34.958881 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:00:34.960098 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:00:34.960104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:00:34.960290 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:34.960325 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 13:00:34.960439 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 13:00:34.960591 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":295,"threads_started":1}
I20260809 13:00:34.962317 8575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1df4621f85c4315abbf260eff069e6e" candidate_uuid: "f99bf84f74b948159d5dfebf961c10ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "390bb3a6078d436898a165c5dd8c8377" is_pre_election: true
I20260809 13:00:34.962940 8575 raft_consensus.cc:2454] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99bf84f74b948159d5dfebf961c10ee in term 0.
I20260809 13:00:34.962939 8432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1df4621f85c4315abbf260eff069e6e" candidate_uuid: "f99bf84f74b948159d5dfebf961c10ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929b08811b7e4e91b3cec796420a52ef" is_pre_election: true
I20260809 13:00:34.963498 8432 raft_consensus.cc:2454] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99bf84f74b948159d5dfebf961c10ee in term 0.
I20260809 13:00:34.963654 8208 leader_election.cc:302] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 390bb3a6078d436898a165c5dd8c8377, f99bf84f74b948159d5dfebf961c10ee; no voters:
I20260809 13:00:34.964103 8649 raft_consensus.cc:2781] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:00:34.964263 8649 raft_consensus.cc:482] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:00:34.964423 8649 raft_consensus.cc:3037] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:00:34.967042 8649 raft_consensus.cc:504] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } } peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } }
I20260809 13:00:34.967620 8649 leader_election.cc:288] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [CANDIDATE]: Term 1 election: Requested vote from peers 390bb3a6078d436898a165c5dd8c8377 (127.21.235.67:44887), 929b08811b7e4e91b3cec796420a52ef (127.21.235.66:46257)
I20260809 13:00:34.968204 8575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1df4621f85c4315abbf260eff069e6e" candidate_uuid: "f99bf84f74b948159d5dfebf961c10ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "390bb3a6078d436898a165c5dd8c8377"
I20260809 13:00:34.968312 8432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1df4621f85c4315abbf260eff069e6e" candidate_uuid: "f99bf84f74b948159d5dfebf961c10ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929b08811b7e4e91b3cec796420a52ef"
I20260809 13:00:34.968457 8575 raft_consensus.cc:3037] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:00:34.968565 8432 raft_consensus.cc:3037] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:00:34.970868 8575 raft_consensus.cc:2454] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99bf84f74b948159d5dfebf961c10ee in term 1.
I20260809 13:00:34.970954 8432 raft_consensus.cc:2454] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99bf84f74b948159d5dfebf961c10ee in term 1.
I20260809 13:00:34.971334 8208 leader_election.cc:302] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 390bb3a6078d436898a165c5dd8c8377, f99bf84f74b948159d5dfebf961c10ee; no voters:
I20260809 13:00:34.971647 8649 raft_consensus.cc:2781] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:00:34.972147 8649 raft_consensus.cc:686] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [term 1 LEADER]: Becoming Leader. State: Replica: f99bf84f74b948159d5dfebf961c10ee, State: Running, Role: LEADER
I20260809 13:00:34.972646 8649 consensus_queue.cc:237] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } } peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } }
I20260809 13:00:34.977435 8123 catalog_manager.cc:5495] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee reported cstate change: term changed from 0 to 1, leader changed from <none> to f99bf84f74b948159d5dfebf961c10ee (127.21.235.65). New cstate: current_term: 1 leader_uuid: "f99bf84f74b948159d5dfebf961c10ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99bf84f74b948159d5dfebf961c10ee" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 41273 } health_report { overall_health: HEALTHY } } }
I20260809 13:00:35.074887 8640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:35.069563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49358 (local address 127.21.235.65:41273)
0809 13:00:35.069748 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:35.069753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:35.069775 (+ 22us) server_negotiation.cc:407] Connection header received
0809 13:00:35.070026 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:35.070031 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:35.070113 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:35.070242 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:00:35.070581 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:35.071389 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:35.072734 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:35.073157 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:35.073258 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:35.073771 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:00:35.074377 (+ 606us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:00:35.074518 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:35.074616 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 13:00:35.074685 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 13:00:35.083278 8432 raft_consensus.cc:1261] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef [term 1 FOLLOWER]: Refusing update from remote peer f99bf84f74b948159d5dfebf961c10ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:00:35.083314 8575 raft_consensus.cc:1261] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377 [term 1 FOLLOWER]: Refusing update from remote peer f99bf84f74b948159d5dfebf961c10ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:00:35.084237 8649 consensus_queue.cc:1035] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "390bb3a6078d436898a165c5dd8c8377" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:00:35.084546 8654 consensus_queue.cc:1035] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "929b08811b7e4e91b3cec796420a52ef" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:00:35.096036 8657 mvcc.cc:205] Tried to move back new op lower bound from 7316604662089940992 to 7316604661652234240. Current Snapshot: MvccSnapshot[applied={T|T < 7316604662089940992}]
I20260809 13:00:35.110798 8123 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:00:35.111039 8123 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:00:35.114864 8123 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: Sending DeleteTablet for 3 replicas of tablet e1df4621f85c4315abbf260eff069e6e
I20260809 13:00:35.116186 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 8164
I20260809 13:00:35.116448 8552 tablet_service.cc:1514] Processing DeleteTablet for tablet e1df4621f85c4315abbf260eff069e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:00:35 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:42346
I20260809 13:00:35.116474 8410 tablet_service.cc:1514] Processing DeleteTablet for tablet e1df4621f85c4315abbf260eff069e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:00:35 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:46718
I20260809 13:00:35.117239 8259 tablet_service.cc:1514] Processing DeleteTablet for tablet e1df4621f85c4315abbf260eff069e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:00:35 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:49350
I20260809 13:00:35.129331 8668 tablet_replica.cc:330] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef: stopping tablet replica
I20260809 13:00:35.129801 8671 tablet_replica.cc:330] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee: stopping tablet replica
I20260809 13:00:35.129886 8668 raft_consensus.cc:2229] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:00:35.130255 8671 raft_consensus.cc:2229] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:00:35.130497 8668 raft_consensus.cc:2258] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:00:35.131289 8671 raft_consensus.cc:2258] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:00:35.131474 8669 tablet_replica.cc:330] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377: stopping tablet replica
I20260809 13:00:35.131995 8669 raft_consensus.cc:2229] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:00:35.132568 8669 raft_consensus.cc:2258] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:00:35.134087 8671 ts_tablet_manager.cc:1905] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:00:35.135077 8668 ts_tablet_manager.cc:1905] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:00:35.135406 8669 ts_tablet_manager.cc:1905] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:00:35.142283 8668 ts_tablet_manager.cc:1918] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:00:35.142953 8668 log.cc:1196] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/e1df4621f85c4315abbf260eff069e6e
I20260809 13:00:35.143148 8669 ts_tablet_manager.cc:1918] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:00:35.143340 8669 log.cc:1196] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/e1df4621f85c4315abbf260eff069e6e
I20260809 13:00:35.143573 8668 ts_tablet_manager.cc:1939] T e1df4621f85c4315abbf260eff069e6e P 929b08811b7e4e91b3cec796420a52ef: Deleting consensus metadata
I20260809 13:00:35.143972 8669 ts_tablet_manager.cc:1939] T e1df4621f85c4315abbf260eff069e6e P 390bb3a6078d436898a165c5dd8c8377: Deleting consensus metadata
I20260809 13:00:35.145632 8107 catalog_manager.cc:4841] TS 929b08811b7e4e91b3cec796420a52ef (127.21.235.66:46257): tablet e1df4621f85c4315abbf260eff069e6e (table test-table [id=127de2d121fb42398ed73676003997a9]) successfully deleted
I20260809 13:00:35.146018 8107 catalog_manager.cc:4841] TS 390bb3a6078d436898a165c5dd8c8377 (127.21.235.67:44887): tablet e1df4621f85c4315abbf260eff069e6e (table test-table [id=127de2d121fb42398ed73676003997a9]) successfully deleted
I20260809 13:00:35.146365 8671 ts_tablet_manager.cc:1918] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:00:35.146631 8671 log.cc:1196] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/e1df4621f85c4315abbf260eff069e6e
I20260809 13:00:35.147333 8671 ts_tablet_manager.cc:1939] T e1df4621f85c4315abbf260eff069e6e P f99bf84f74b948159d5dfebf961c10ee: Deleting consensus metadata
I20260809 13:00:35.149233 8108 catalog_manager.cc:4841] TS f99bf84f74b948159d5dfebf961c10ee (127.21.235.65:41273): tablet e1df4621f85c4315abbf260eff069e6e (table test-table [id=127de2d121fb42398ed73676003997a9]) successfully deleted
I20260809 13:00:35.160322 8640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:35.117870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49374 (local address 127.21.235.65:41273)
0809 13:00:35.118063 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:35.118070 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:35.143011 (+ 24941us) server_negotiation.cc:407] Connection header received
0809 13:00:35.143259 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:35.143268 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:35.143383 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:35.143521 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:35.146391 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:35.147928 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:35.149971 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:35.150574 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:35.150724 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:35.153546 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:35.153568 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:35.153595 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:35.153655 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:35.157039 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:35.157786 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:35.157801 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:35.157803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:35.158014 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:35.158462 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:35.158470 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:35.158474 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:35.158944 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:35.159239 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:35.159662 (+ 423us) server_negotiation.cc:299] Negotiation successful
0809 13:00:35.159925 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 13:00:35.161039 8327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:00:35.261557 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 8164
I20260809 13:00:35.285851 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 8337
I20260809 13:00:35.299360 8642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:35.286770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46724 (local address 127.21.235.66:46257)
0809 13:00:35.286971 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:35.286976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:35.286995 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:00:35.287134 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:35.287138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:35.287201 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:35.287288 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:35.288707 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:35.289456 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:35.290903 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:35.291301 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:35.291400 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:35.293816 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:35.293830 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:35.293844 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:35.293896 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:35.296781 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:35.297399 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:35.297407 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:35.297409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:35.297542 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:35.297860 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:35.297863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:35.297865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:35.298228 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:35.298504 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:35.298819 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 13:00:35.298992 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 13:00:35.300096 8475 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:00:35.389503 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 8337
I20260809 13:00:35.409816 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 8483
I20260809 13:00:35.424166 8638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:35.410741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42360 (local address 127.21.235.67:44887)
0809 13:00:35.410955 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 13:00:35.410962 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:00:35.411062 (+ 100us) server_negotiation.cc:407] Connection header received
0809 13:00:35.411278 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:00:35.411283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:00:35.411349 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:00:35.411494 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:00:35.412946 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:35.413770 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:35.415252 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:00:35.415656 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:00:35.415794 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:00:35.418312 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:00:35.418327 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:00:35.418343 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:00:35.418386 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:00:35.421204 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:35.421896 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:35.421906 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:35.421909 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:35.422063 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:00:35.422717 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:00:35.422720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:00:35.422722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:00:35.423076 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:00:35.423350 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 13:00:35.423734 (+ 384us) server_negotiation.cc:299] Negotiation successful
0809 13:00:35.423904 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 13:00:35.424947 8620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:00:35.514117 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 8483
I20260809 13:00:35.533761 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 8085
I20260809 13:00:35.534989 8156 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:00:35.650683 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 8085
I20260809 13:00:55.677335 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:42617
--webserver_interface=127.21.235.126
--webserver_port=40127
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:46303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:42617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:00:56.029323 8676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:00:56.029699 8676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:00:56.029863 8676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:00:56.037384 8676 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 13:00:56.037539 8676 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 13:00:56.037635 8676 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 13:00:56.037734 8676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:00:56.037844 8676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:00:56.037968 8676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:00:56.038058 8676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:00:56.038141 8676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:00:56.048418 8676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:46303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:42617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:42617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=40127
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:00:56.049883 8676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:00:56.051338 8676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:00:56.064414 8683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:56.064342 8686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:00:56.064730 8676 server_base.cc:1034] running on GCE node
W20260809 13:00:56.065099 8684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:00:56.066045 8676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:00:56.067739 8676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:00:56.069015 8676 hybrid_clock.cc:648] HybridClock initialized: now 1786280456068926 us; error 69 us; skew 500 ppm
I20260809 13:00:56.069471 8676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:00:56 dist-test-slave-gp6n krb5kdc[7467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280456, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:00:56.073118 8676 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 13:00:56.074822 8676 webserver.cc:460] Webserver started at http://127.21.235.126:40127/ using document root <none> and password file <none>
I20260809 13:00:56.075587 8676 fs_manager.cc:359] Metadata directory not provided
I20260809 13:00:56.075733 8676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:00:56.081835 8676 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 13:00:56.085216 8693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:56.087010 8676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:00:56.087224 8676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "6b7e946c4e184052afd2f872047d8e42"
format_stamp: "Formatted at 2026-08-09 13:00:09 on dist-test-slave-gp6n"
server_key: "12c95c171f36288dc31c1a6cc4842007"
server_key_iv: "2ec67b21f6cf591f52199be3f9c4f3ce"
server_key_version: "encryptionkey@0"
I20260809 13:00:56.088402 8676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:00:56.102487 8676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:00:56.107595 8676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:00:56.107951 8676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:00:56.129617 8676 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:42617
I20260809 13:00:56.129642 8750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:42617 every 8 connection(s)
I20260809 13:00:56.131469 8676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:00:56.138433 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 8676
I20260809 13:00:56.139110 8751 sys_catalog.cc:263] Verifying existing consensus state
I20260809 13:00:56.139611 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:41273
--local_ip_for_outbound_sockets=127.21.235.65
--tserver_master_addrs=127.21.235.126:42617
--webserver_port=45979
--webserver_interface=127.21.235.65
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:46303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:00:56.144383 8751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: Bootstrap starting.
I20260809 13:00:56.167580 8751 log.cc:824] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: Log is configured to *not* fsync() on all Append() calls
I20260809 13:00:56.183938 8751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 13:00:56.184514 8751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: Bootstrap complete.
I20260809 13:00:56.194077 8751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7e946c4e184052afd2f872047d8e42" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42617 } }
I20260809 13:00:56.195043 8751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b7e946c4e184052afd2f872047d8e42, State: Initialized, Role: FOLLOWER
I20260809 13:00:56.195662 8751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7e946c4e184052afd2f872047d8e42" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42617 } }
I20260809 13:00:56.195899 8751 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:00:56.196022 8751 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:00:56.196182 8751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 2 FOLLOWER]: Advancing to term 3
I20260809 13:00:56.200094 8751 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7e946c4e184052afd2f872047d8e42" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42617 } }
I20260809 13:00:56.200608 8751 leader_election.cc:302] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b7e946c4e184052afd2f872047d8e42; no voters:
I20260809 13:00:56.201337 8751 leader_election.cc:288] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 13:00:56.201476 8755 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 13:00:56.202729 8755 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [term 3 LEADER]: Becoming Leader. State: Replica: 6b7e946c4e184052afd2f872047d8e42, State: Running, Role: LEADER
I20260809 13:00:56.203380 8755 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7e946c4e184052afd2f872047d8e42" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42617 } }
I20260809 13:00:56.203617 8751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:00:56.207943 8756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6b7e946c4e184052afd2f872047d8e42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7e946c4e184052afd2f872047d8e42" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42617 } } }
I20260809 13:00:56.208339 8756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [sys.catalog]: This master's current role is: LEADER
I20260809 13:00:56.208963 8757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b7e946c4e184052afd2f872047d8e42. Latest consensus state: current_term: 3 leader_uuid: "6b7e946c4e184052afd2f872047d8e42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b7e946c4e184052afd2f872047d8e42" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 42617 } } }
I20260809 13:00:56.209249 8757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42 [sys.catalog]: This master's current role is: LEADER
I20260809 13:00:56.226387 8763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:00:56.236662 8763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:00:56.238305 8763 catalog_manager.cc:1259] Loaded cluster ID: a31abf667e9f4c52aebd5855c2b155f8
I20260809 13:00:56.238526 8763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:00:56.243175 8763 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:00:56.244832 8763 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: Loaded TSK: 0
I20260809 13:00:56.245066 8763 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6b7e946c4e184052afd2f872047d8e42: Loaded TSK: 1
F20260809 13:00:56.246232 8763 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe37b0e49ad google::LogMessage::Fail() at ??:0
@ 0x7fe37b0e788a google::LogMessage::SendToLog() at ??:0
@ 0x7fe37b0e44de google::LogMessage::Flush() at ??:0
@ 0x7fe37b0e5a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
I20260809 13:00:56.256474 8773 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7fe37ee487d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fe37ee3773e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fe390b131c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fe390b12783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fe390b9e2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fe390b9e0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fe390be85e9 std::function<>::operator()() at ??:0
@ 0x7fe390b16cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fe390b14c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe390b9c610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fe390b9c439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe37dd30345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe37dd3391c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fe37dd33799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe37dd0eeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe380fe86db start_thread at ??:0
@ 0x7fe379f7e61f clone at ??:0
W20260809 13:00:56.553706 8753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:00:56.554041 8753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:00:56.554203 8753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:00:56.561868 8753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:00:56.561982 8753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:00:56.562197 8753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 13:00:56.572273 8753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:46303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:41273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=45979
--tserver_master_addrs=127.21.235.126:42617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:00:56.573669 8753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:00:56.575160 8753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:00:56.589062 8803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:56.589085 8805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:56.589023 8807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:00:56.589970 8753 server_base.cc:1034] running on GCE node
I20260809 13:00:56.590761 8753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:00:56.591794 8753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:00:56.593060 8753 hybrid_clock.cc:648] HybridClock initialized: now 1786280456593001 us; error 49 us; skew 500 ppm
I20260809 13:00:56.593441 8753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:00:56 dist-test-slave-gp6n krb5kdc[7467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280456, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:00:56.596825 8753 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 13:00:56.598263 8753 webserver.cc:460] Webserver started at http://127.21.235.65:45979/ using document root <none> and password file <none>
I20260809 13:00:56.599032 8753 fs_manager.cc:359] Metadata directory not provided
I20260809 13:00:56.599140 8753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:00:56.604671 8753 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 13:00:56.607542 8814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:56.608901 8753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:00:56.609035 8753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "f99bf84f74b948159d5dfebf961c10ee"
format_stamp: "Formatted at 2026-08-09 13:00:10 on dist-test-slave-gp6n"
server_key: "c6727ef2bc6b1b70e3a1d3451fe8116f"
server_key_iv: "d84bf348ff324e47609313cef9a57d41"
server_key_version: "encryptionkey@0"
I20260809 13:00:56.610044 8753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:00:56.633546 8753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:00:56.637692 8753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:00:56.637923 8753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:00:56.638923 8753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:00:56.640309 8753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:00:56.640407 8753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:56.640503 8753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:00:56.640561 8753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:56.676810 8753 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:41273
I20260809 13:00:56.676932 8934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:41273 every 8 connection(s)
I20260809 13:00:56.678382 8753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 13:00:56.683372 8937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:56.681467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:56701)
0809 13:00:56.682184 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:56.682505 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":280,"threads_started":1}
W20260809 13:00:56.684013 8935 heartbeater.cc:644] Failed to heartbeat to 127.21.235.126:42617 (0 consecutive failures): Network error: Failed to ping master at 127.21.235.126:42617: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
I20260809 13:00:56.684811 8937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:56.684506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:46307)
0809 13:00:56.684636 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:56.684704 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 13:00:56.685616 8937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:56.685322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:58489)
0809 13:00:56.685439 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:56.685511 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 13:00:56.685878 8935 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 13:00:56.687952 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 8753
I20260809 13:00:56.688583 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:46257
--local_ip_for_outbound_sockets=127.21.235.66
--tserver_master_addrs=127.21.235.126:42617
--webserver_port=37971
--webserver_interface=127.21.235.66
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:46303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:00:57.030467 8938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:00:57.030757 8938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:00:57.030944 8938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:00:57.038146 8938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:00:57.038250 8938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:00:57.038489 8938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 13:00:57.048442 8938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:46303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:46257
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=37971
--tserver_master_addrs=127.21.235.126:42617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:00:57.049820 8938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:00:57.051190 8938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:00:57.064404 8943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:57.064487 8946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:57.064548 8944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:00:57.065152 8938 server_base.cc:1034] running on GCE node
I20260809 13:00:57.065704 8938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:00:57.066689 8938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:00:57.067885 8938 hybrid_clock.cc:648] HybridClock initialized: now 1786280457067832 us; error 53 us; skew 500 ppm
I20260809 13:00:57.068198 8938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:00:57 dist-test-slave-gp6n krb5kdc[7467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280457, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:00:57.071597 8938 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 13:00:57.073033 8938 webserver.cc:460] Webserver started at http://127.21.235.66:37971/ using document root <none> and password file <none>
I20260809 13:00:57.073727 8938 fs_manager.cc:359] Metadata directory not provided
I20260809 13:00:57.073871 8938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:00:57.079312 8938 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:00:57.082084 8953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:57.083472 8938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:00:57.083650 8938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "929b08811b7e4e91b3cec796420a52ef"
format_stamp: "Formatted at 2026-08-09 13:00:10 on dist-test-slave-gp6n"
server_key: "b6ba40033a26cf8ec5130ffb473161ab"
server_key_iv: "dfedc33a3e3196d88210cc82350c8c7c"
server_key_version: "encryptionkey@0"
I20260809 13:00:57.084689 8938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:00:57.100183 8938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:00:57.104732 8938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:00:57.105028 8938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:00:57.106029 8938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:00:57.107517 8938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:00:57.107641 8938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:57.107777 8938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:00:57.107877 8938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:57.143253 8938 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:46257
I20260809 13:00:57.143345 9074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:46257 every 8 connection(s)
I20260809 13:00:57.144785 8938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 13:00:57.146843 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 8938
I20260809 13:00:57.147522 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:44887
--local_ip_for_outbound_sockets=127.21.235.67
--tserver_master_addrs=127.21.235.126:42617
--webserver_port=37681
--webserver_interface=127.21.235.67
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:46303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:00:57.150626 9077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:57.148074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:60425)
0809 13:00:57.148981 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:57.149308 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":312,"threads_started":1}
W20260809 13:00:57.151311 9075 heartbeater.cc:644] Failed to heartbeat to 127.21.235.126:42617 (0 consecutive failures): Network error: Failed to ping master at 127.21.235.126:42617: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
I20260809 13:00:57.152323 9077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:57.151882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:44207)
0809 13:00:57.152076 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:57.152171 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 13:00:57.153448 9077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:57.152994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:50057)
0809 13:00:57.153171 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:57.153276 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 13:00:57.153781 9075 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 13:00:57.503054 9078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:00:57.503418 9078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:00:57.503587 9078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:00:57.510948 9078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:00:57.511102 9078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:00:57.511270 9078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 13:00:57.521240 9078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:46303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:44887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=37681
--tserver_master_addrs=127.21.235.126:42617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:00:57.522723 9078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:00:57.524057 9078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:00:57.537209 9083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:57.537271 9086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:00:57.537294 9084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:00:57.538368 9078 server_base.cc:1034] running on GCE node
I20260809 13:00:57.539052 9078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:00:57.539990 9078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:00:57.541184 9078 hybrid_clock.cc:648] HybridClock initialized: now 1786280457541122 us; error 32 us; skew 500 ppm
I20260809 13:00:57.541591 9078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:00:57 dist-test-slave-gp6n krb5kdc[7467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280457, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:00:57.544811 9078 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 13:00:57.546171 9078 webserver.cc:460] Webserver started at http://127.21.235.67:37681/ using document root <none> and password file <none>
I20260809 13:00:57.546901 9078 fs_manager.cc:359] Metadata directory not provided
I20260809 13:00:57.547049 9078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:00:57.552310 9078 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 13:00:57.555027 9093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:57.556353 9078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:00:57.556527 9078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "390bb3a6078d436898a165c5dd8c8377"
format_stamp: "Formatted at 2026-08-09 13:00:11 on dist-test-slave-gp6n"
server_key: "6c662aa6a0db5f02d02b1fc92309573d"
server_key_iv: "766ef75f8bd3ad689e67ae4e325729c3"
server_key_version: "encryptionkey@0"
I20260809 13:00:57.557512 9078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:00:57.583110 9078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:00:57.587757 9078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:00:57.588172 9078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:00:57.589260 9078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:00:57.590721 9078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:00:57.590850 9078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:57.590973 9078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:00:57.591061 9078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:00:57.625273 9078 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:44887
I20260809 13:00:57.625331 9220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:44887 every 8 connection(s)
I20260809 13:00:57.626888 9078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 13:00:57.629186 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 9078
I20260809 13:00:57.632869 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:57.630241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:54977)
0809 13:00:57.631775 (+ 1534us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:57.632031 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1239,"thread_start_us":1027,"threads_started":1}
W20260809 13:00:57.633527 9222 heartbeater.cc:644] Failed to heartbeat to 127.21.235.126:42617 (0 consecutive failures): Network error: Failed to ping master at 127.21.235.126:42617: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
I20260809 13:00:57.634300 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:57.634008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:50611)
0809 13:00:57.634135 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:57.634199 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 13:00:57.635195 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:57.634845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:56509)
0809 13:00:57.634989 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:57.635073 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 13:00:57.635447 9222 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 13:00:57.687551 9228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:57.686578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:38989)
0809 13:00:57.687155 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:57.687301 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":323,"threads_started":1}
I20260809 13:00:58.155196 9229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:58.154350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:54627)
0809 13:00:58.154861 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:58.154967 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":265,"threads_started":1}
I20260809 13:00:58.636806 9230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:58.635994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:51237)
0809 13:00:58.636485 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:58.636572 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":233,"threads_started":1}
I20260809 13:00:58.689296 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:58.688509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:36707)
0809 13:00:58.688958 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:58.689042 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":232,"threads_started":1}
I20260809 13:00:59.156801 9232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:59.156098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:54077)
0809 13:00:59.156565 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:59.156643 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":242,"threads_started":1}
I20260809 13:00:59.638639 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:59.637811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:48131)
0809 13:00:59.638320 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:59.638440 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":232,"threads_started":1}
I20260809 13:00:59.690861 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:00:59.690129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:53131)
0809 13:00:59.690603 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 13:00:59.690682 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":224,"threads_started":1}
I20260809 13:01:00.158802 9235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:00.157768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:36743)
0809 13:01:00.158507 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:00.158600 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":450,"threads_started":1}
I20260809 13:01:00.640185 9236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:00.639489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:49525)
0809 13:01:00.639925 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:00.640006 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":233,"threads_started":1}
I20260809 13:01:00.692565 9237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:00.691828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:57185)
0809 13:01:00.692296 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:00.692384 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":244,"threads_started":1}
I20260809 13:01:01.160589 9238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:01.159833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:49437)
0809 13:01:01.160279 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:01.160394 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":224,"threads_started":1}
I20260809 13:01:01.641884 9240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:01.641165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:52053)
0809 13:01:01.641624 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:01.641708 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":231,"threads_started":1}
I20260809 13:01:01.694595 9243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:01.693674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:53607)
0809 13:01:01.694263 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:01.694350 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":345,"threads_started":1}
I20260809 13:01:02.162637 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:02.161629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:39737)
0809 13:01:02.162278 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:02.162438 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":251,"threads_started":1}
I20260809 13:01:02.643780 9245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:02.642954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:34661)
0809 13:01:02.643509 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:02.643594 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":302,"threads_started":1}
I20260809 13:01:02.696481 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:02.695615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:55949)
0809 13:01:02.696147 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:02.696256 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":305,"threads_started":1}
I20260809 13:01:03.164367 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:03.163597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:41975)
0809 13:01:03.164093 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:03.164193 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":256,"threads_started":1}
I20260809 13:01:03.645488 9248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:03.644758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:41395)
0809 13:01:03.645193 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:03.645283 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":231,"threads_started":1}
I20260809 13:01:03.698302 9249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:03.697493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:56365)
0809 13:01:03.697997 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:03.698088 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":219,"threads_started":1}
I20260809 13:01:04.166335 9250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:04.165424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:51981)
0809 13:01:04.165990 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:04.166092 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":235,"threads_started":1}
I20260809 13:01:04.647337 9251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:04.646567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:53127)
0809 13:01:04.647042 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:04.647124 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":226,"threads_started":1}
I20260809 13:01:04.700064 9252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:04.699320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:49293)
0809 13:01:04.699801 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:04.699889 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":262,"threads_started":1}
I20260809 13:01:05.168277 9253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:05.167473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:37427)
0809 13:01:05.167997 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:05.168096 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":254,"threads_started":1}
I20260809 13:01:05.649047 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:05.648267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:42821)
0809 13:01:05.648752 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:05.648843 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":262,"threads_started":1}
I20260809 13:01:05.701699 9255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:05.700968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:44961)
0809 13:01:05.701450 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:05.701528 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":245,"threads_started":1}
I20260809 13:01:06.170127 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:06.169310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:40751)
0809 13:01:06.169857 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:06.169944 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":282,"threads_started":1}
I20260809 13:01:06.650826 9258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:06.649996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:59137)
0809 13:01:06.650553 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:06.650641 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":248,"threads_started":1}
I20260809 13:01:06.703442 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:06.702716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:37197)
0809 13:01:06.703175 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:06.703256 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":264,"threads_started":1}
I20260809 13:01:07.172101 9260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:07.171320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:37289)
0809 13:01:07.171823 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:07.171925 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":273,"threads_started":1}
I20260809 13:01:07.652576 9261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:07.651863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:40247)
0809 13:01:07.652329 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:07.652412 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":279,"threads_started":1}
I20260809 13:01:07.705106 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:07.704381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:39847)
0809 13:01:07.704824 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:07.704921 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":229,"threads_started":1}
I20260809 13:01:08.173923 9263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:08.173201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:60831)
0809 13:01:08.173647 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:08.173738 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":222,"threads_started":1}
I20260809 13:01:08.654177 9264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:08.653477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:49907)
0809 13:01:08.653903 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:08.653999 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":226,"threads_started":1}
I20260809 13:01:08.706821 9265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:08.706083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:34717)
0809 13:01:08.706577 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:08.706665 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":220,"threads_started":1}
I20260809 13:01:09.175833 9267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:09.175142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:42171)
0809 13:01:09.175589 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:09.175676 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":227,"threads_started":1}
I20260809 13:01:09.656011 9268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:09.655287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:47699)
0809 13:01:09.655756 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:09.655840 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":249,"threads_started":1}
I20260809 13:01:09.708554 9269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:09.707763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:53141)
0809 13:01:09.708246 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:09.708344 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":246,"threads_started":1}
I20260809 13:01:10.177711 9271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:10.176887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:51431)
0809 13:01:10.177406 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:10.177511 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":257,"threads_started":1}
I20260809 13:01:10.657737 9272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:10.656951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:42613)
0809 13:01:10.657444 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:10.657545 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":244,"threads_started":1}
I20260809 13:01:10.710289 9273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:10.709470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:35549)
0809 13:01:10.710011 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:10.710098 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":306,"threads_started":1}
I20260809 13:01:11.179325 9274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:11.178644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:44385)
0809 13:01:11.179065 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:11.179144 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":211,"threads_started":1}
I20260809 13:01:11.659499 9275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:11.658697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:59951)
0809 13:01:11.659141 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:11.659250 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":223,"threads_started":1}
I20260809 13:01:11.712155 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:11.711423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.65:42481)
0809 13:01:11.711856 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:11.711968 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":234,"threads_started":1}
I20260809 13:01:12.181308 9277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:12.180513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.66:60691)
0809 13:01:12.181013 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:12.181118 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":258,"threads_started":1}
I20260809 13:01:12.633708 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 8753
I20260809 13:01:12.656919 9278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:12.634890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50166 (local address 127.21.235.65:41273)
0809 13:01:12.635584 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:12.635608 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:12.635665 (+ 57us) server_negotiation.cc:407] Connection header received
0809 13:01:12.637587 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:12.637597 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:12.638035 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:12.638341 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:12.639888 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:12.641178 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:12.642588 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:12.643289 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:12.643431 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:12.650276 (+ 6845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:12.650285 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:12.650305 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:12.650357 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:12.653682 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:12.654375 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:12.654382 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:12.654388 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:12.654643 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:12.654993 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:12.654996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:12.654998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:12.655548 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:12.655780 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:12.656177 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 13:01:12.656629 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":292,"threads_started":1}
I20260809 13:01:12.658623 8930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:12.661710 9281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:12.660795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:42617 (local address 127.21.235.67:49203)
0809 13:01:12.661308 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:12.661478 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.126:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":281,"threads_started":1}
I20260809 13:01:12.747875 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 8753
I20260809 13:01:12.769608 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 8938
I20260809 13:01:12.792891 9283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:12.770640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51398 (local address 127.21.235.66:46257)
0809 13:01:12.771193 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:12.771208 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:12.771246 (+ 38us) server_negotiation.cc:407] Connection header received
0809 13:01:12.772904 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:12.772914 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:12.773353 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:12.773651 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:12.775211 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:12.777102 (+ 1891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:12.779191 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:12.780506 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:12.780722 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:12.784232 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:12.784249 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:12.784278 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:12.784356 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:12.788924 (+ 4568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:12.789936 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:12.789943 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:12.789949 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:12.790140 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:12.790648 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:12.790652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:12.790655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:12.791278 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:12.791637 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:12.792124 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 13:01:12.792605 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":264,"threads_started":1}
I20260809 13:01:12.795003 9070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:12.889119 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 8938
I20260809 13:01:12.910249 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 9078
I20260809 13:01:12.928598 9286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:12.911279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60816 (local address 127.21.235.67:44887)
0809 13:01:12.911912 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:12.911926 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:12.911971 (+ 45us) server_negotiation.cc:407] Connection header received
0809 13:01:12.913667 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:12.913676 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:12.914121 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:12.914594 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:12.915807 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:12.917058 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:12.918344 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:12.919063 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:12.919209 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:12.922161 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:12.922168 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:12.922184 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:12.922220 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:12.925278 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:12.926054 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:12.926061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:12.926068 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:12.926248 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:12.926675 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:12.926679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:12.926682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:12.927233 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:12.927492 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:12.927845 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 13:01:12.928278 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":312,"threads_started":1}
I20260809 13:01:12.929986 9216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:13.017246 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 9078
2026-08-09T13:01:13Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65391 ms)
[----------] 30 tests from SecurityITest (309254 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:01:13 dist-test-slave-gp6n krb5kdc[9290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:01:13 dist-test-slave-gp6n krb5kdc[9290](info): set up 2 sockets
Aug 09 13:01:13 dist-test-slave-gp6n krb5kdc[9290](info): commencing operation
krb5kdc: starting...
W20260809 13:01:14.223592 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:01:14 dist-test-slave-gp6n krb5kdc[9290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:01:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:01:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 13:01:14.539831 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:45233
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:45233
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:14.890012 9305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:14.890276 9305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:14.890498 9305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:14.897550 9305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:01:14.897677 9305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:14.897761 9305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:14.897857 9305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:01:14.897943 9305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:01:14.907648 9305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:45233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:45233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:14.908933 9305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:14.910290 9305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:14.922884 9310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:14.922963 9311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:14.923486 9314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:14.923852 9305 server_base.cc:1034] running on GCE node
I20260809 13:01:14.924661 9305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:14.926040 9305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:14.927302 9305 hybrid_clock.cc:648] HybridClock initialized: now 1786280474927263 us; error 49 us; skew 500 ppm
I20260809 13:01:14.927687 9305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:14 dist-test-slave-gp6n krb5kdc[9290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280474, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:14.930948 9305 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 13:01:14.932447 9305 webserver.cc:460] Webserver started at http://127.21.235.126:37687/ using document root <none> and password file <none>
I20260809 13:01:14.933178 9305 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:14.933320 9305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:14.933655 9305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:14.936847 9305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "433853906d474bbfbe57315258c4cffc"
format_stamp: "Formatted at 2026-08-09 13:01:14 on dist-test-slave-gp6n"
server_key: "2a7823ae1e45e4d7504696a54c5d5158"
server_key_iv: "6aa8b959a3dac51e76117fd5dd8231ea"
server_key_version: "encryptionkey@0"
I20260809 13:01:14.937723 9305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "433853906d474bbfbe57315258c4cffc"
format_stamp: "Formatted at 2026-08-09 13:01:14 on dist-test-slave-gp6n"
server_key: "2a7823ae1e45e4d7504696a54c5d5158"
server_key_iv: "6aa8b959a3dac51e76117fd5dd8231ea"
server_key_version: "encryptionkey@0"
I20260809 13:01:14.943192 9305 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 13:01:14.946682 9322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:14.948016 9305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:01:14.948205 9305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "433853906d474bbfbe57315258c4cffc"
format_stamp: "Formatted at 2026-08-09 13:01:14 on dist-test-slave-gp6n"
server_key: "2a7823ae1e45e4d7504696a54c5d5158"
server_key_iv: "6aa8b959a3dac51e76117fd5dd8231ea"
server_key_version: "encryptionkey@0"
I20260809 13:01:14.948415 9305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:14.966375 9305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:14.970934 9305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:14.971243 9305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:14.990880 9305 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:45233
I20260809 13:01:14.990887 9379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:45233 every 8 connection(s)
I20260809 13:01:14.992322 9305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:01:14.996333 9381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:14.998572 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 9305
I20260809 13:01:14.998817 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:01:14.999676 22445 external_mini_cluster.cc:1428] Setting key 00520984346fcefd7a6cbc8f66777b72
I20260809 13:01:15.009964 9381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc: Bootstrap starting.
I20260809 13:01:15.014074 9381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:15.015404 9381 log.cc:824] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc: Log is configured to *not* fsync() on all Append() calls
Aug 09 13:01:15 dist-test-slave-gp6n krb5kdc[9290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:01:15.018687 9381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc: No bootstrap required, opened a new log
I20260809 13:01:15.024565 9384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:15.001871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41078 (local address 127.21.235.126:45233)
0809 13:01:15.002801 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:15.002817 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:15.002863 (+ 46us) server_negotiation.cc:407] Connection header received
0809 13:01:15.004637 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:15.004661 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:15.005078 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:15.005404 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:15.007380 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:15.008700 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:15.010695 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:15.011405 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:15.011561 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:15.017114 (+ 5553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:15.017123 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:15.017137 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:15.017173 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:15.020421 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:15.021185 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:15.021192 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:15.021198 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:15.021375 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:15.021803 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:15.021807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:15.021809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:15.022339 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:15.022710 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:15.023170 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 13:01:15.023579 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":465,"threads_started":1}
I20260809 13:01:15.026775 9381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433853906d474bbfbe57315258c4cffc" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 45233 } }
I20260809 13:01:15.027199 9381 raft_consensus.cc:374] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:15.027318 9381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 433853906d474bbfbe57315258c4cffc, State: Initialized, Role: FOLLOWER
I20260809 13:01:15.027936 9381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433853906d474bbfbe57315258c4cffc" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 45233 } }
I20260809 13:01:15.028195 9381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:01:15.028332 9381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:01:15.028487 9381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:15.031358 9381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433853906d474bbfbe57315258c4cffc" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 45233 } }
I20260809 13:01:15.031849 9381 leader_election.cc:302] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 433853906d474bbfbe57315258c4cffc; no voters:
I20260809 13:01:15.032456 9381 leader_election.cc:288] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:01:15.032609 9386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:15.033437 9386 raft_consensus.cc:686] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc [term 1 LEADER]: Becoming Leader. State: Replica: 433853906d474bbfbe57315258c4cffc, State: Running, Role: LEADER
I20260809 13:01:15.034053 9386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433853906d474bbfbe57315258c4cffc" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 45233 } }
I20260809 13:01:15.035027 9381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:01:15.037067 9388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 433853906d474bbfbe57315258c4cffc. Latest consensus state: current_term: 1 leader_uuid: "433853906d474bbfbe57315258c4cffc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433853906d474bbfbe57315258c4cffc" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 45233 } } }
I20260809 13:01:15.037271 9387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "433853906d474bbfbe57315258c4cffc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433853906d474bbfbe57315258c4cffc" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 45233 } } }
I20260809 13:01:15.037591 9387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:15.037601 9388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:15.039232 9391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:01:15.044282 9391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:01:15.053535 9391 catalog_manager.cc:1347] Generated new cluster ID: fdf1267bfd6a4631b1fc6b10d9191f61
I20260809 13:01:15.053654 9391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:01:15.081337 9391 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:01:15.082898 9391 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:01:15.089727 9391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc: Generated new TSK 0
I20260809 13:01:15.090759 9391 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
I20260809 13:01:15.175304 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:45233
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:15.522168 9409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:15.522501 9409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:15.522701 9409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:15.530153 9409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:15.530251 9409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:15.530529 9409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 13:01:15.540328 9409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:45233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:15.541582 9409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:15.542907 9409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:15.556980 9414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:15.556871 9417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:15.556924 9415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:15.557843 9409 server_base.cc:1034] running on GCE node
I20260809 13:01:15.558538 9409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:15.559526 9409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:15.560798 9409 hybrid_clock.cc:648] HybridClock initialized: now 1786280475560710 us; error 61 us; skew 500 ppm
I20260809 13:01:15.561147 9409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:15 dist-test-slave-gp6n krb5kdc[9290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280475, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:15.564606 9409 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 13:01:15.566097 9409 webserver.cc:460] Webserver started at http://127.21.235.65:39307/ using document root <none> and password file <none>
I20260809 13:01:15.566900 9409 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:15.567051 9409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:15.567420 9409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:15.570649 9409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "6982cc4a16bc471e8bd496a3beb0d8ad"
format_stamp: "Formatted at 2026-08-09 13:01:15 on dist-test-slave-gp6n"
server_key: "ee236d016ab67bdf2b974a6375aa0b4c"
server_key_iv: "46928da1f33d26b2cba7971c2c29e6e7"
server_key_version: "encryptionkey@0"
I20260809 13:01:15.571529 9409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "6982cc4a16bc471e8bd496a3beb0d8ad"
format_stamp: "Formatted at 2026-08-09 13:01:15 on dist-test-slave-gp6n"
server_key: "ee236d016ab67bdf2b974a6375aa0b4c"
server_key_iv: "46928da1f33d26b2cba7971c2c29e6e7"
server_key_version: "encryptionkey@0"
I20260809 13:01:15.577382 9409 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:01:15.581265 9425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:15.582849 9409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:01:15.583051 9409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "6982cc4a16bc471e8bd496a3beb0d8ad"
format_stamp: "Formatted at 2026-08-09 13:01:15 on dist-test-slave-gp6n"
server_key: "ee236d016ab67bdf2b974a6375aa0b4c"
server_key_iv: "46928da1f33d26b2cba7971c2c29e6e7"
server_key_version: "encryptionkey@0"
I20260809 13:01:15.583298 9409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:15.602567 9409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:15.607112 9409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:15.607430 9409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:15.608467 9409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:01:15.609903 9409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:01:15.610011 9409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:15.610183 9409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:01:15.610260 9409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:15.646561 9409 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:45749
I20260809 13:01:15.646622 9552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:45749 every 8 connection(s)
I20260809 13:01:15.647902 9409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 13:01:15.656198 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 9409
I20260809 13:01:15.656433 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 13:01:15.657214 22445 external_mini_cluster.cc:1428] Setting key c409472b409c51f501bd60495f802166
Aug 09 13:01:15 dist-test-slave-gp6n krb5kdc[9290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280475, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:01:15.670207 9555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:15.650620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:57501 (local address 127.21.235.126:45233)
0809 13:01:15.651105 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:15.651110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:15.652026 (+ 916us) server_negotiation.cc:407] Connection header received
0809 13:01:15.653888 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:15.653894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:15.654000 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:15.654127 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:15.655709 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:15.656890 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:15.658882 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:15.659473 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:15.659606 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:15.663311 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:15.663320 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:15.663327 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:15.663371 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:15.666914 (+ 3543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:15.667653 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:15.667662 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:15.667669 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:15.667837 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:15.668236 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:15.668243 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:15.668248 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:15.668620 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:15.668773 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:15.669459 (+ 686us) server_negotiation.cc:299] Negotiation successful
0809 13:01:15.669725 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":265,"threads_started":1}
I20260809 13:01:15.670804 9556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:15.651143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:45233 (local address 127.21.235.65:57501)
0809 13:01:15.651855 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:15.651895 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:15.653614 (+ 1719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:15.654381 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:15.654389 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:15.654970 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:01:15.655535 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:15.655551 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:15.657050 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:15.657054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:15.658744 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:15.658758 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:15.659689 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:15.659694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:15.659991 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:15.660673 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:01:15.660699 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:01:15.663085 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:01:15.667050 (+ 3965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:01:15.667056 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:01:15.667068 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:01:15.667511 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:01:15.667946 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:01:15.667949 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:01:15.667951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:01:15.668120 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:01:15.668739 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:01:15.668744 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:01:15.669044 (+ 300us) client_negotiation.cc:769] Sending connection context
0809 13:01:15.669278 (+ 234us) client_negotiation.cc:240] Negotiation successful
0809 13:01:15.669670 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":268,"threads_started":1}
I20260809 13:01:15.672438 9553 heartbeater.cc:347] Connected to a master server at 127.21.235.126:45233
I20260809 13:01:15.672750 9553 heartbeater.cc:464] Registering TS with master...
I20260809 13:01:15.673480 9553 heartbeater.cc:511] Master 127.21.235.126:45233 requested a full tablet report, sending...
I20260809 13:01:15.675623 9340 ts_manager.cc:194] Registered new tserver with Master: 6982cc4a16bc471e8bd496a3beb0d8ad (127.21.235.65:45749)
I20260809 13:01:15.677045 9340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:57501
WARNING: no policy specified for kudu/127.21.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
I20260809 13:01:15.725394 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:45233
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:16.066429 9559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:16.066779 9559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:16.066967 9559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:16.074296 9559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:16.074489 9559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:16.074671 9559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 13:01:16.084707 9559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:45233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:16.086107 9559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:16.087747 9559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:16.101042 9564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:16.101315 9565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:16.101897 9568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:16.102660 9559 server_base.cc:1034] running on GCE node
I20260809 13:01:16.103327 9559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:16.104364 9559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:16.105603 9559 hybrid_clock.cc:648] HybridClock initialized: now 1786280476105568 us; error 60 us; skew 500 ppm
I20260809 13:01:16.105926 9559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:16 dist-test-slave-gp6n krb5kdc[9290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:16.109364 9559 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 13:01:16.110948 9559 webserver.cc:460] Webserver started at http://127.21.235.66:39797/ using document root <none> and password file <none>
I20260809 13:01:16.111649 9559 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:16.111805 9559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:16.112138 9559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:16.115152 9559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "9e01f02c26b74cb69e0bdc0bd7166ad5"
format_stamp: "Formatted at 2026-08-09 13:01:16 on dist-test-slave-gp6n"
server_key: "cf2dd5f7e93561307d07d80eb523f22a"
server_key_iv: "96b118ead76590b3972598b606c1affc"
server_key_version: "encryptionkey@0"
I20260809 13:01:16.115960 9559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "9e01f02c26b74cb69e0bdc0bd7166ad5"
format_stamp: "Formatted at 2026-08-09 13:01:16 on dist-test-slave-gp6n"
server_key: "cf2dd5f7e93561307d07d80eb523f22a"
server_key_iv: "96b118ead76590b3972598b606c1affc"
server_key_version: "encryptionkey@0"
I20260809 13:01:16.121775 9559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:01:16.125423 9575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:16.126776 9559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:01:16.126957 9559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "9e01f02c26b74cb69e0bdc0bd7166ad5"
format_stamp: "Formatted at 2026-08-09 13:01:16 on dist-test-slave-gp6n"
server_key: "cf2dd5f7e93561307d07d80eb523f22a"
server_key_iv: "96b118ead76590b3972598b606c1affc"
server_key_version: "encryptionkey@0"
I20260809 13:01:16.127161 9559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:16.145510 9559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:16.150265 9559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:16.150617 9559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:16.151736 9559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:01:16.153303 9559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:01:16.153431 9559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:16.153620 9559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:01:16.153700 9559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:16.189575 9559 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:38165
I20260809 13:01:16.189626 9702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:38165 every 8 connection(s)
I20260809 13:01:16.191000 9559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 13:01:16.194855 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 9559
I20260809 13:01:16.195060 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 13:01:16.195654 22445 external_mini_cluster.cc:1428] Setting key e507ffddc31f4b1a572df2249f09d800
Aug 09 13:01:16 dist-test-slave-gp6n krb5kdc[9290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:01:16.214612 9705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:16.193728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:34755 (local address 127.21.235.126:45233)
0809 13:01:16.194208 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:16.194213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:16.195509 (+ 1296us) server_negotiation.cc:407] Connection header received
0809 13:01:16.198145 (+ 2636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:16.198151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:16.198235 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:16.198355 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:16.200101 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.201264 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.202975 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.203717 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.203866 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:16.207532 (+ 3666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:16.207542 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:16.207549 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:16.207595 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:16.211128 (+ 3533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:16.212120 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:16.212124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:16.212126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:16.212254 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:16.212787 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:16.212790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:16.212792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:16.213061 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:16.213173 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:16.214079 (+ 906us) server_negotiation.cc:299] Negotiation successful
0809 13:01:16.214249 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":230,"threads_started":1}
I20260809 13:01:16.216156 9706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:16.194285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:45233 (local address 127.21.235.66:34755)
0809 13:01:16.195319 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:16.195371 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:16.197794 (+ 2423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:16.198640 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:16.198648 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:16.199257 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:01:16.199864 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:16.199894 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.201452 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.201456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:16.202816 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:16.202829 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.203938 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.203943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:16.204137 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:16.204923 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:01:16.204960 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:01:16.207254 (+ 2294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:01:16.211276 (+ 4022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:01:16.211287 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:01:16.211306 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:01:16.211982 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:01:16.212370 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:01:16.212377 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:01:16.212383 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:01:16.212660 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:01:16.213193 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:01:16.213203 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:01:16.213667 (+ 464us) client_negotiation.cc:769] Sending connection context
0809 13:01:16.213982 (+ 315us) client_negotiation.cc:240] Negotiation successful
0809 13:01:16.214594 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":371,"threads_started":1}
I20260809 13:01:16.218052 9703 heartbeater.cc:347] Connected to a master server at 127.21.235.126:45233
I20260809 13:01:16.218467 9703 heartbeater.cc:464] Registering TS with master...
I20260809 13:01:16.219483 9703 heartbeater.cc:511] Master 127.21.235.126:45233 requested a full tablet report, sending...
I20260809 13:01:16.221556 9340 ts_manager.cc:194] Registered new tserver with Master: 9e01f02c26b74cb69e0bdc0bd7166ad5 (127.21.235.66:38165)
I20260809 13:01:16.222786 9340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:34755
WARNING: no policy specified for kudu/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
I20260809 13:01:16.270279 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:45233
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:16.611054 9709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:16.611379 9709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:16.611557 9709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:16.618674 9709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:16.618826 9709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:16.619015 9709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 13:01:16.628773 9709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:45233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:16.630088 9709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:16.631464 9709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:16.646515 9714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:16.646539 9715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:16.645473 9717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:16.647037 9709 server_base.cc:1034] running on GCE node
I20260809 13:01:16.647868 9709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:16.649010 9709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:16.650246 9709 hybrid_clock.cc:648] HybridClock initialized: now 1786280476650169 us; error 63 us; skew 500 ppm
I20260809 13:01:16.650622 9709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:16 dist-test-slave-gp6n krb5kdc[9290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:16.654292 9709 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 13:01:16.655905 9709 webserver.cc:460] Webserver started at http://127.21.235.67:33373/ using document root <none> and password file <none>
I20260809 13:01:16.656610 9709 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:16.656761 9709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:16.657094 9709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:16.660077 9709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "71ec2f902ff64f0090431180789bf4e4"
format_stamp: "Formatted at 2026-08-09 13:01:16 on dist-test-slave-gp6n"
server_key: "dc3819d7f035a94b9a491bb8ec2b8049"
server_key_iv: "2617df8d435a8d0d588b16e5626f681e"
server_key_version: "encryptionkey@0"
I20260809 13:01:16.660861 9709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "71ec2f902ff64f0090431180789bf4e4"
format_stamp: "Formatted at 2026-08-09 13:01:16 on dist-test-slave-gp6n"
server_key: "dc3819d7f035a94b9a491bb8ec2b8049"
server_key_iv: "2617df8d435a8d0d588b16e5626f681e"
server_key_version: "encryptionkey@0"
I20260809 13:01:16.666172 9709 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 13:01:16.669654 9726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:16.671002 9709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:01:16.671176 9709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "71ec2f902ff64f0090431180789bf4e4"
format_stamp: "Formatted at 2026-08-09 13:01:16 on dist-test-slave-gp6n"
server_key: "dc3819d7f035a94b9a491bb8ec2b8049"
server_key_iv: "2617df8d435a8d0d588b16e5626f681e"
server_key_version: "encryptionkey@0"
I20260809 13:01:16.671389 9709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:16.679943 9553 heartbeater.cc:503] Master 127.21.235.126:45233 was elected leader, sending a full tablet report...
I20260809 13:01:16.703156 9709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:16.707466 9709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:16.707767 9709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:16.708758 9709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:01:16.710104 9709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:01:16.710213 9709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:16.710374 9709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:01:16.710484 9709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 13:01:16.744823 9709 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:39845
I20260809 13:01:16.744870 9847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:39845 every 8 connection(s)
I20260809 13:01:16.746194 9709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 13:01:16.750641 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 9709
I20260809 13:01:16.750836 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 13:01:16.751289 22445 external_mini_cluster.cc:1428] Setting key f61233fdda1f8361b0633192c601aa63
Aug 09 13:01:16 dist-test-slave-gp6n krb5kdc[9290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:01:16.767176 9850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:16.748897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:58319 (local address 127.21.235.126:45233)
0809 13:01:16.749388 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:16.749395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:16.750496 (+ 1101us) server_negotiation.cc:407] Connection header received
0809 13:01:16.752909 (+ 2413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:16.752913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:16.752981 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:16.753115 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:16.754865 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.755663 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.757223 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.757627 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.757767 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:16.761163 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:16.761168 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:16.761172 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:16.761204 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:16.764035 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:16.764758 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:16.764763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:16.764766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:16.764886 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:16.765375 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:16.765379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:16.765382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:16.765664 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:16.765843 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:16.766576 (+ 733us) server_negotiation.cc:299] Negotiation successful
0809 13:01:16.766797 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":221,"threads_started":1}
I20260809 13:01:16.768158 9851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:16.749575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:45233 (local address 127.21.235.67:58319)
0809 13:01:16.750307 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:16.750353 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:16.752513 (+ 2160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:16.753401 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:16.753412 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:16.753965 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:01:16.754697 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:16.754716 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.755829 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.755834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:16.757073 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:16.757085 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.757864 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.757870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:16.758108 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:16.758784 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:01:16.758806 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:01:16.760959 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:01:16.764165 (+ 3206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:01:16.764171 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:01:16.764183 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:01:16.764611 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:01:16.765019 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:01:16.765024 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:01:16.765030 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:01:16.765239 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:01:16.765807 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:01:16.765814 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:01:16.766149 (+ 335us) client_negotiation.cc:769] Sending connection context
0809 13:01:16.766405 (+ 256us) client_negotiation.cc:240] Negotiation successful
0809 13:01:16.766932 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":292,"threads_started":1}
I20260809 13:01:16.769459 9848 heartbeater.cc:347] Connected to a master server at 127.21.235.126:45233
I20260809 13:01:16.769763 9848 heartbeater.cc:464] Registering TS with master...
I20260809 13:01:16.770551 9848 heartbeater.cc:511] Master 127.21.235.126:45233 requested a full tablet report, sending...
I20260809 13:01:16.771925 9340 ts_manager.cc:194] Registered new tserver with Master: 71ec2f902ff64f0090431180789bf4e4 (127.21.235.67:39845)
I20260809 13:01:16.772689 9340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:58319
I20260809 13:01:16.778765 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:01:16.796002 9850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:16.783467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41094 (local address 127.21.235.126:45233)
0809 13:01:16.783634 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:16.783638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:16.784055 (+ 417us) server_negotiation.cc:407] Connection header received
0809 13:01:16.784184 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:16.784188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:16.784245 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:16.784327 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:16.785695 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.786388 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.787949 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.788444 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.788536 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:16.791066 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:16.791071 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:16.791074 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:16.791103 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:16.793542 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:16.794134 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:16.794137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:16.794140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:16.794262 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:16.794753 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:16.794757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:16.794759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:16.795084 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:16.795265 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:16.795606 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 13:01:16.795773 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 13:01:16.800485 9340 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41094:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:01:16.808100 9340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:01:16.850092 9868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:16.822235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52372 (local address 127.21.235.66:38165)
0809 13:01:16.838097 (+ 15862us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:16.838111 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:16.840657 (+ 2546us) server_negotiation.cc:407] Connection header received
0809 13:01:16.840973 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:16.840982 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:16.841199 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:16.841378 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:16.842360 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.844084 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.847303 (+ 3219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.848787 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.848954 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:16.849169 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:16.849579 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 13:01:16.849744 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15636,"thread_start_us":15493,"threads_started":1}
I20260809 13:01:16.850169 9863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:16.820688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:39845 (local address 127.0.0.1:33028)
0809 13:01:16.836968 (+ 16280us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:16.837031 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:16.837295 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:16.842989 (+ 5694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:16.842994 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:16.843021 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:16.843349 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:16.843360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.845414 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.845423 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:16.847170 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:16.847184 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.849320 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.849328 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:16.849567 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:16.849594 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 13:01:16.849652 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 13:01:16.849755 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14987,"mutex_wait_us":58,"thread_start_us":15066,"threads_started":1}
I20260809 13:01:16.850219 9866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:16.821468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:38165 (local address 127.0.0.1:52372)
0809 13:01:16.840535 (+ 19067us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:16.840552 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:16.840763 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:16.841676 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:16.841680 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:16.841706 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:16.842145 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:16.842160 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.844296 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.844302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:16.847102 (+ 2800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:16.847130 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.849001 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.849015 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:16.849225 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:16.849317 (+ 92us) client_negotiation.cc:769] Sending connection context
0809 13:01:16.849515 (+ 198us) client_negotiation.cc:240] Negotiation successful
0809 13:01:16.849716 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18937,"mutex_wait_us":9,"thread_start_us":18794,"threads_started":1}
I20260809 13:01:16.851151 9864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:16.821069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:45749 (local address 127.0.0.1:42692)
0809 13:01:16.838698 (+ 17629us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:16.838731 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:16.838889 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:16.841250 (+ 2361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:16.841257 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:16.841289 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:16.841676 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:16.841687 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.843511 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.843517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:16.847159 (+ 3642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:16.847173 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.849363 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.849369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:16.850503 (+ 1134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:16.850553 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 13:01:16.850645 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 13:01:16.850786 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17464,"mutex_wait_us":1066,"thread_start_us":17249,"threads_started":1}
I20260809 13:01:16.851373 9867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:16.822135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42692 (local address 127.21.235.65:45749)
0809 13:01:16.840228 (+ 18093us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:16.840239 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:16.840277 (+ 38us) server_negotiation.cc:407] Connection header received
0809 13:01:16.840396 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:16.840405 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:16.840704 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:16.840911 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:16.841890 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.843349 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.847361 (+ 4012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.849026 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.849183 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:16.849467 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:16.850815 (+ 1348us) server_negotiation.cc:299] Negotiation successful
0809 13:01:16.851009 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17909,"thread_start_us":17730,"threads_started":1}
I20260809 13:01:16.852010 9865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:16.821061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33028 (local address 127.21.235.67:39845)
0809 13:01:16.842359 (+ 21298us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:16.842368 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:16.842429 (+ 61us) server_negotiation.cc:407] Connection header received
0809 13:01:16.842542 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:16.842550 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:16.842781 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:16.842998 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:16.843579 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.845261 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.847362 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.849026 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.850537 (+ 1511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:16.850900 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:16.850975 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 13:01:16.851135 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21080,"thread_start_us":20887,"threads_started":1}
I20260809 13:01:16.854789 9629 tablet_service.cc:1467] Processing CreateTablet for tablet 83ea358b8adb4e6c88f8b6227f082002 (DEFAULT_TABLE table=test-table [id=ff8439fb9c9545d28f25ec5c3a21c0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:01:16.855098 9479 tablet_service.cc:1467] Processing CreateTablet for tablet 83ea358b8adb4e6c88f8b6227f082002 (DEFAULT_TABLE table=test-table [id=ff8439fb9c9545d28f25ec5c3a21c0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:01:16.856364 9629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83ea358b8adb4e6c88f8b6227f082002. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:16.856510 9479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83ea358b8adb4e6c88f8b6227f082002. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:16.856527 9777 tablet_service.cc:1467] Processing CreateTablet for tablet 83ea358b8adb4e6c88f8b6227f082002 (DEFAULT_TABLE table=test-table [id=ff8439fb9c9545d28f25ec5c3a21c0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:01:16.857915 9777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83ea358b8adb4e6c88f8b6227f082002. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:16.868780 9869 tablet_bootstrap.cc:492] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad: Bootstrap starting.
I20260809 13:01:16.871949 9870 tablet_bootstrap.cc:492] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5: Bootstrap starting.
I20260809 13:01:16.873404 9869 tablet_bootstrap.cc:654] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:16.873654 9871 tablet_bootstrap.cc:492] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4: Bootstrap starting.
I20260809 13:01:16.875047 9869 log.cc:824] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:16.876049 9870 tablet_bootstrap.cc:654] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:16.876912 9871 tablet_bootstrap.cc:654] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:16.877307 9870 log.cc:824] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:16.878068 9871 log.cc:824] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:16.878700 9869 tablet_bootstrap.cc:492] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad: No bootstrap required, opened a new log
I20260809 13:01:16.879005 9869 ts_tablet_manager.cc:1397] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260809 13:01:16.880821 9870 tablet_bootstrap.cc:492] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5: No bootstrap required, opened a new log
I20260809 13:01:16.881088 9870 ts_tablet_manager.cc:1397] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 13:01:16.885090 9871 tablet_bootstrap.cc:492] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4: No bootstrap required, opened a new log
I20260809 13:01:16.885335 9871 ts_tablet_manager.cc:1397] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260809 13:01:16.890887 9869 raft_consensus.cc:348] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ec2f902ff64f0090431180789bf4e4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39845 } } peers { permanent_uuid: "6982cc4a16bc471e8bd496a3beb0d8ad" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45749 } } peers { permanent_uuid: "9e01f02c26b74cb69e0bdc0bd7166ad5" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38165 } }
I20260809 13:01:16.891356 9869 raft_consensus.cc:374] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:16.891464 9869 raft_consensus.cc:729] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6982cc4a16bc471e8bd496a3beb0d8ad, State: Initialized, Role: FOLLOWER
I20260809 13:01:16.892122 9869 consensus_queue.cc:260] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ec2f902ff64f0090431180789bf4e4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39845 } } peers { permanent_uuid: "6982cc4a16bc471e8bd496a3beb0d8ad" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45749 } } peers { permanent_uuid: "9e01f02c26b74cb69e0bdc0bd7166ad5" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38165 } }
I20260809 13:01:16.892961 9871 raft_consensus.cc:348] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ec2f902ff64f0090431180789bf4e4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39845 } } peers { permanent_uuid: "6982cc4a16bc471e8bd496a3beb0d8ad" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45749 } } peers { permanent_uuid: "9e01f02c26b74cb69e0bdc0bd7166ad5" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38165 } }
I20260809 13:01:16.893361 9871 raft_consensus.cc:374] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:16.893460 9871 raft_consensus.cc:729] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71ec2f902ff64f0090431180789bf4e4, State: Initialized, Role: FOLLOWER
I20260809 13:01:16.894080 9869 ts_tablet_manager.cc:1428] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260809 13:01:16.894124 9871 consensus_queue.cc:260] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ec2f902ff64f0090431180789bf4e4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39845 } } peers { permanent_uuid: "6982cc4a16bc471e8bd496a3beb0d8ad" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45749 } } peers { permanent_uuid: "9e01f02c26b74cb69e0bdc0bd7166ad5" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38165 } }
I20260809 13:01:16.894383 9870 raft_consensus.cc:348] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ec2f902ff64f0090431180789bf4e4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39845 } } peers { permanent_uuid: "6982cc4a16bc471e8bd496a3beb0d8ad" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45749 } } peers { permanent_uuid: "9e01f02c26b74cb69e0bdc0bd7166ad5" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38165 } }
I20260809 13:01:16.894893 9870 raft_consensus.cc:374] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:16.895022 9870 raft_consensus.cc:729] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e01f02c26b74cb69e0bdc0bd7166ad5, State: Initialized, Role: FOLLOWER
I20260809 13:01:16.895433 9848 heartbeater.cc:503] Master 127.21.235.126:45233 was elected leader, sending a full tablet report...
I20260809 13:01:16.895649 9870 consensus_queue.cc:260] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ec2f902ff64f0090431180789bf4e4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39845 } } peers { permanent_uuid: "6982cc4a16bc471e8bd496a3beb0d8ad" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45749 } } peers { permanent_uuid: "9e01f02c26b74cb69e0bdc0bd7166ad5" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38165 } }
I20260809 13:01:16.895995 9871 ts_tablet_manager.cc:1428] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260809 13:01:16.897877 9703 heartbeater.cc:503] Master 127.21.235.126:45233 was elected leader, sending a full tablet report...
I20260809 13:01:16.898588 9870 ts_tablet_manager.cc:1428] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5: Time spent starting tablet: real 0.017s user 0.007s sys 0.011s
W20260809 13:01:16.900368 9554 tablet.cc:2369] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:01:16.905373 9875 raft_consensus.cc:482] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:01:16.905593 9875 raft_consensus.cc:504] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ec2f902ff64f0090431180789bf4e4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39845 } } peers { permanent_uuid: "6982cc4a16bc471e8bd496a3beb0d8ad" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45749 } } peers { permanent_uuid: "9e01f02c26b74cb69e0bdc0bd7166ad5" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38165 } }
I20260809 13:01:16.907657 9875 leader_election.cc:288] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71ec2f902ff64f0090431180789bf4e4 (127.21.235.67:39845), 9e01f02c26b74cb69e0bdc0bd7166ad5 (127.21.235.66:38165)
I20260809 13:01:16.914537 9865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:16.908076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:55019 (local address 127.21.235.67:39845)
0809 13:01:16.908263 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:16.908271 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:16.908979 (+ 708us) server_negotiation.cc:407] Connection header received
0809 13:01:16.909259 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:16.909280 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:16.909364 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:16.909450 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:16.910152 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.910940 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.912782 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.913557 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.913746 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:16.913825 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:16.914107 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 13:01:16.914231 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 13:01:16.914593 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:16.908150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:39845 (local address 127.21.235.65:55019)
0809 13:01:16.908852 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:16.908875 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:16.909066 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:16.909513 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:16.909519 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:16.909552 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:16.910032 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:16.910046 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.911087 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.911095 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:16.912628 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:16.912642 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.913771 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.913778 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:16.913934 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:16.913959 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 13:01:16.914080 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 13:01:16.914209 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":393,"threads_started":1}
I20260809 13:01:16.915414 9880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:16.908176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:38165 (local address 127.21.235.65:35637)
0809 13:01:16.909573 (+ 1397us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:16.909594 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:16.909754 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:16.910057 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:16.910061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:16.910093 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:16.910497 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:16.910509 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.911629 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.911635 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:16.913201 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:16.913216 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.914506 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.914513 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:16.914777 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:16.914840 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 13:01:16.915002 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 13:01:16.915147 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1266,"thread_start_us":986,"threads_started":1}
I20260809 13:01:16.915450 9868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:16.908021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:35637 (local address 127.21.235.66:38165)
0809 13:01:16.908239 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:16.908245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:16.909693 (+ 1448us) server_negotiation.cc:407] Connection header received
0809 13:01:16.909890 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:16.909893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:16.909944 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:16.910008 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:16.910678 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.911465 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.913378 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.914215 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.914414 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:16.914534 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:16.915047 (+ 513us) server_negotiation.cc:299] Negotiation successful
0809 13:01:16.915182 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 13:01:16.915668 9799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83ea358b8adb4e6c88f8b6227f082002" candidate_uuid: "6982cc4a16bc471e8bd496a3beb0d8ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71ec2f902ff64f0090431180789bf4e4" is_pre_election: true
I20260809 13:01:16.916249 9799 raft_consensus.cc:2454] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6982cc4a16bc471e8bd496a3beb0d8ad in term 0.
I20260809 13:01:16.916548 9650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83ea358b8adb4e6c88f8b6227f082002" candidate_uuid: "6982cc4a16bc471e8bd496a3beb0d8ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e01f02c26b74cb69e0bdc0bd7166ad5" is_pre_election: true
I20260809 13:01:16.916940 9429 leader_election.cc:302] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6982cc4a16bc471e8bd496a3beb0d8ad, 71ec2f902ff64f0090431180789bf4e4; no voters:
I20260809 13:01:16.917152 9650 raft_consensus.cc:2454] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6982cc4a16bc471e8bd496a3beb0d8ad in term 0.
I20260809 13:01:16.917335 9875 raft_consensus.cc:2781] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:01:16.917505 9875 raft_consensus.cc:482] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:01:16.917616 9875 raft_consensus.cc:3037] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:16.919943 9875 raft_consensus.cc:504] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ec2f902ff64f0090431180789bf4e4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39845 } } peers { permanent_uuid: "6982cc4a16bc471e8bd496a3beb0d8ad" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45749 } } peers { permanent_uuid: "9e01f02c26b74cb69e0bdc0bd7166ad5" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38165 } }
I20260809 13:01:16.920627 9875 leader_election.cc:288] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [CANDIDATE]: Term 1 election: Requested vote from peers 71ec2f902ff64f0090431180789bf4e4 (127.21.235.67:39845), 9e01f02c26b74cb69e0bdc0bd7166ad5 (127.21.235.66:38165)
I20260809 13:01:16.921087 9799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83ea358b8adb4e6c88f8b6227f082002" candidate_uuid: "6982cc4a16bc471e8bd496a3beb0d8ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71ec2f902ff64f0090431180789bf4e4"
I20260809 13:01:16.921118 9650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83ea358b8adb4e6c88f8b6227f082002" candidate_uuid: "6982cc4a16bc471e8bd496a3beb0d8ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e01f02c26b74cb69e0bdc0bd7166ad5"
I20260809 13:01:16.921252 9799 raft_consensus.cc:3037] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:16.921341 9650 raft_consensus.cc:3037] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:16.924031 9799 raft_consensus.cc:2454] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6982cc4a16bc471e8bd496a3beb0d8ad in term 1.
I20260809 13:01:16.924208 9650 raft_consensus.cc:2454] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6982cc4a16bc471e8bd496a3beb0d8ad in term 1.
I20260809 13:01:16.924577 9429 leader_election.cc:302] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6982cc4a16bc471e8bd496a3beb0d8ad, 71ec2f902ff64f0090431180789bf4e4; no voters:
I20260809 13:01:16.924860 9875 raft_consensus.cc:2781] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:16.926296 9875 raft_consensus.cc:686] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [term 1 LEADER]: Becoming Leader. State: Replica: 6982cc4a16bc471e8bd496a3beb0d8ad, State: Running, Role: LEADER
I20260809 13:01:16.926816 9875 consensus_queue.cc:237] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ec2f902ff64f0090431180789bf4e4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39845 } } peers { permanent_uuid: "6982cc4a16bc471e8bd496a3beb0d8ad" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45749 } } peers { permanent_uuid: "9e01f02c26b74cb69e0bdc0bd7166ad5" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38165 } }
I20260809 13:01:16.931645 9340 catalog_manager.cc:5495] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 6982cc4a16bc471e8bd496a3beb0d8ad (127.21.235.65). New cstate: current_term: 1 leader_uuid: "6982cc4a16bc471e8bd496a3beb0d8ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71ec2f902ff64f0090431180789bf4e4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6982cc4a16bc471e8bd496a3beb0d8ad" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 45749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e01f02c26b74cb69e0bdc0bd7166ad5" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38165 } health_report { overall_health: UNKNOWN } } }
W20260809 13:01:16.943048 9704 tablet.cc:2369] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:01:16.955631 9867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:16.950120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42696 (local address 127.21.235.65:45749)
0809 13:01:16.950307 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:16.950313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:16.950385 (+ 72us) server_negotiation.cc:407] Connection header received
0809 13:01:16.950668 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:16.950672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:16.950776 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:16.950908 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:01:16.951209 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.951953 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.953478 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:16.953920 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:16.954036 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:16.954575 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:01:16.955101 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:01:16.955230 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:16.955341 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 13:01:16.955413 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 13:01:16.964564 9799 raft_consensus.cc:1261] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4 [term 1 FOLLOWER]: Refusing update from remote peer 6982cc4a16bc471e8bd496a3beb0d8ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:01:16.964566 9650 raft_consensus.cc:1261] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5 [term 1 FOLLOWER]: Refusing update from remote peer 6982cc4a16bc471e8bd496a3beb0d8ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:01:16.965588 9881 consensus_queue.cc:1035] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "71ec2f902ff64f0090431180789bf4e4" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:01:16.966013 9875 consensus_queue.cc:1035] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e01f02c26b74cb69e0bdc0bd7166ad5" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 38165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:01:16.978554 9885 mvcc.cc:205] Tried to move back new op lower bound from 7316604833635528704 to 7316604833496354816. Current Snapshot: MvccSnapshot[applied={T|T < 7316604833635528704}]
I20260809 13:01:16.994771 9339 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:01:16.995112 9339 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:01:16.999248 9339 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 433853906d474bbfbe57315258c4cffc: Sending DeleteTablet for 3 replicas of tablet 83ea358b8adb4e6c88f8b6227f082002
I20260809 13:01:17.000428 9479 tablet_service.cc:1514] Processing DeleteTablet for tablet 83ea358b8adb4e6c88f8b6227f082002 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:01:16 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:42692
I20260809 13:01:17.000418 9777 tablet_service.cc:1514] Processing DeleteTablet for tablet 83ea358b8adb4e6c88f8b6227f082002 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:01:16 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:33028
I20260809 13:01:17.003816 9629 tablet_service.cc:1514] Processing DeleteTablet for tablet 83ea358b8adb4e6c88f8b6227f082002 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:01:16 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:52372
I20260809 13:01:17.006071 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 9409
I20260809 13:01:17.023001 9900 tablet_replica.cc:330] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad: stopping tablet replica
I20260809 13:01:17.023587 9900 raft_consensus.cc:2229] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:01:17.024395 9900 raft_consensus.cc:2258] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:01:17.025326 9899 tablet_replica.cc:330] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4: stopping tablet replica
I20260809 13:01:17.025851 9899 raft_consensus.cc:2229] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:01:17.026652 9899 raft_consensus.cc:2258] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 09 13:01:17 dist-test-slave-gp6n krb5kdc[9290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 13:01:17.027519 9901 tablet_replica.cc:330] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5: stopping tablet replica
I20260809 13:01:17.028075 9901 raft_consensus.cc:2229] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:01:17.028657 9901 raft_consensus.cc:2258] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:01:17.029194 9899 ts_tablet_manager.cc:1905] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:01:17.031291 9901 ts_tablet_manager.cc:1905] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:01:17.032543 9900 ts_tablet_manager.cc:1905] T 83ea358b8adb4e6c88f8b6227f082002 P 6982cc4a16bc471e8bd496a3beb0d8ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:01:17.036659 9867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:17.007484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42700 (local address 127.21.235.65:45749)
0809 13:01:17.007684 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:17.007691 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:17.011692 (+ 4001us) server_negotiation.cc:407] Connection header received
0809 13:01:17.012574 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:17.012584 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:17.012722 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:17.012812 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:17.017295 (+ 4483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:17.018786 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:17.021355 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:17.022113 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:17.022252 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:17.028159 (+ 5907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:17.028174 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:17.028192 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:17.028243 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:17.032556 (+ 4313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:17.033606 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:17.033615 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:17.033620 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:17.033867 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:17.034428 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:17.034433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:17.034436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:17.034935 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:17.035247 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:17.035898 (+ 651us) server_negotiation.cc:299] Negotiation successful
0809 13:01:17.036192 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 13:01:17.038357 9547 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:17.039239 9901 ts_tablet_manager.cc:1918] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:01:17.039474 9901 log.cc:1196] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/83ea358b8adb4e6c88f8b6227f082002
I20260809 13:01:17.040673 9901 ts_tablet_manager.cc:1939] T 83ea358b8adb4e6c88f8b6227f082002 P 9e01f02c26b74cb69e0bdc0bd7166ad5: Deleting consensus metadata
I20260809 13:01:17.042882 9323 catalog_manager.cc:4841] TS 9e01f02c26b74cb69e0bdc0bd7166ad5 (127.21.235.66:38165): tablet 83ea358b8adb4e6c88f8b6227f082002 (table test-table [id=ff8439fb9c9545d28f25ec5c3a21c0fa]) successfully deleted
I20260809 13:01:17.044754 9899 ts_tablet_manager.cc:1918] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:01:17.044956 9899 log.cc:1196] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/83ea358b8adb4e6c88f8b6227f082002
I20260809 13:01:17.045486 9899 ts_tablet_manager.cc:1939] T 83ea358b8adb4e6c88f8b6227f082002 P 71ec2f902ff64f0090431180789bf4e4: Deleting consensus metadata
I20260809 13:01:17.047168 9326 catalog_manager.cc:4841] TS 71ec2f902ff64f0090431180789bf4e4 (127.21.235.67:39845): tablet 83ea358b8adb4e6c88f8b6227f082002 (table test-table [id=ff8439fb9c9545d28f25ec5c3a21c0fa]) successfully deleted
I20260809 13:01:17.141703 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 9409
I20260809 13:01:17.164057 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 9559
W20260809 13:01:17.164038 9324 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.235.65:45749 (error 108)
I20260809 13:01:17.165899 9866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:17.165470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:45749 (local address 127.0.0.1:42704)
0809 13:01:17.165637 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:17.165752 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:45749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 13:01:17.166121 9324 catalog_manager.cc:4584] TS 6982cc4a16bc471e8bd496a3beb0d8ad (127.21.235.65:45749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 83ea358b8adb4e6c88f8b6227f082002: Network error: Client connection negotiation failed: client connection to 127.21.235.65:45749: connect: Connection refused (error 111)
Aug 09 13:01:17 dist-test-slave-gp6n krb5kdc[9290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.66@KRBTEST.COM
I20260809 13:01:17.180517 9868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:17.164889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52380 (local address 127.21.235.66:38165)
0809 13:01:17.165107 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:17.165113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:17.165211 (+ 98us) server_negotiation.cc:407] Connection header received
0809 13:01:17.165474 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:17.165484 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:17.165548 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:17.165665 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:17.167202 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:17.167944 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:17.169230 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:17.169636 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:17.169783 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:17.174253 (+ 4470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:17.174268 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:17.174282 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:17.174323 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:17.177291 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:17.178065 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:17.178073 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:17.178075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:17.178201 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:17.178660 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:17.178666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:17.178670 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:17.179062 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:17.179377 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:17.179703 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 13:01:17.179924 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260809 13:01:17.181180 9698 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:17.219676 9866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:17.219187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:45749 (local address 127.0.0.1:42718)
0809 13:01:17.219407 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:17.219498 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:45749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 13:01:17.220902 9866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:17.220460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:45749 (local address 127.0.0.1:42726)
0809 13:01:17.220653 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:17.220763 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:45749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 13:01:17.270352 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 9559
I20260809 13:01:17.281916 9866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:17.281154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:45749 (local address 127.0.0.1:42742)
0809 13:01:17.281563 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:17.281679 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:45749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194}
I20260809 13:01:17.283402 9866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:17.282870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:45749 (local address 127.0.0.1:42744)
0809 13:01:17.283140 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:17.283235 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:45749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260809 13:01:17.290364 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 9709
Aug 09 13:01:17 dist-test-slave-gp6n krb5kdc[9290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.67@KRBTEST.COM
I20260809 13:01:17.305653 9865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:17.291163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33038 (local address 127.21.235.67:39845)
0809 13:01:17.291367 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:17.291373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:17.291394 (+ 21us) server_negotiation.cc:407] Connection header received
0809 13:01:17.291599 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:17.291604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:17.291680 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:17.291788 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:17.293129 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:17.293880 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:17.295212 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:17.295640 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:17.295774 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:17.300305 (+ 4531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:17.300320 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:17.300336 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:17.300406 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:17.303177 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:17.303806 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:17.303813 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:17.303816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:17.303946 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:17.304274 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:17.304277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:17.304279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:17.304622 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:17.304846 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:17.305216 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 13:01:17.305406 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 13:01:17.306300 9842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:17.364243 9866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:17.363651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:45749 (local address 127.0.0.1:42756)
0809 13:01:17.363923 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:17.364046 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:45749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260809 13:01:17.365460 9866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:17.364977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:45749 (local address 127.0.0.1:42762)
0809 13:01:17.365208 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:17.365300 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:45749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260809 13:01:17.397733 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 9709
I20260809 13:01:17.417425 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 9305
I20260809 13:01:17.419093 9375 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:17.533152 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 9305
2026-08-09T13:01:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4523 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T13:01:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:01:17Z Disabled control of system clock
I20260809 13:01:17.629768 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:35861
--webserver_interface=127.21.235.126
--webserver_port=0
--builtin_ntp_servers=127.21.235.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:35861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 13:01:17.977206 9912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:17.977511 9912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:17.977685 9912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:17.985069 9912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:01:17.985196 9912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:17.985265 9912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:17.985381 9912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:01:17.985466 9912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:01:17.995074 9912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:35861
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:35861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:17.996465 9912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:17.998314 9912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:18.010972 9918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:18.010968 9919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:18.010912 9921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:18.012087 9912 server_base.cc:1034] running on GCE node
I20260809 13:01:18.012816 9912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:18.014199 9912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:18.015412 9912 hybrid_clock.cc:648] HybridClock initialized: now 1786280478015353 us; error 49 us; skew 500 ppm
I20260809 13:01:18.015736 9912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:01:18.023602 9912 webserver.cc:460] Webserver started at http://127.21.235.126:43863/ using document root <none> and password file <none>
I20260809 13:01:18.024322 9912 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:18.024415 9912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:18.024685 9912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:18.027766 9912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "432bada5074140bf97a004319ab9fff8"
format_stamp: "Formatted at 2026-08-09 13:01:18 on dist-test-slave-gp6n"
server_key: "492fd6049ad1dccc372b83309a68dbe9"
server_key_iv: "000e27afe6d93c5278fb11707cbb8a91"
server_key_version: "encryptionkey@0"
I20260809 13:01:18.028615 9912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "432bada5074140bf97a004319ab9fff8"
format_stamp: "Formatted at 2026-08-09 13:01:18 on dist-test-slave-gp6n"
server_key: "492fd6049ad1dccc372b83309a68dbe9"
server_key_iv: "000e27afe6d93c5278fb11707cbb8a91"
server_key_version: "encryptionkey@0"
I20260809 13:01:18.033986 9912 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 13:01:18.037643 9927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:18.039080 9912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:01:18.039279 9912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "432bada5074140bf97a004319ab9fff8"
format_stamp: "Formatted at 2026-08-09 13:01:18 on dist-test-slave-gp6n"
server_key: "492fd6049ad1dccc372b83309a68dbe9"
server_key_iv: "000e27afe6d93c5278fb11707cbb8a91"
server_key_version: "encryptionkey@0"
I20260809 13:01:18.039505 9912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:18.065311 9912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:18.066370 9912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:18.066771 9912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:18.087139 9912 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:35861
I20260809 13:01:18.087162 9983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:35861 every 8 connection(s)
I20260809 13:01:18.088582 9912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:01:18.092614 9984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:18.096382 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 9912
I20260809 13:01:18.096601 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:01:18.097090 22445 external_mini_cluster.cc:1428] Setting key 6305fc2eb0fbf6e61d01a91ab042f1c3
I20260809 13:01:18.105860 9984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8: Bootstrap starting.
I20260809 13:01:18.109652 9984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:18.110939 9984 log.cc:824] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:18.113907 9987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:18.099262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40508 (local address 127.21.235.126:35861)
0809 13:01:18.100435 (+ 1173us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:18.100451 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:18.100506 (+ 55us) server_negotiation.cc:407] Connection header received
0809 13:01:18.103839 (+ 3333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:18.103870 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:18.104480 (+ 610us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:18.104943 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:18.105493 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:18.107525 (+ 2032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:18.108900 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:18.110051 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:18.110265 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:18.111574 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:18.111582 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:18.111597 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:18.112163 (+ 566us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:18.112243 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:18.112265 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:18.112411 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:18.112657 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 13:01:18.112960 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":373,"threads_started":1}
I20260809 13:01:18.114071 9984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8: No bootstrap required, opened a new log
I20260809 13:01:18.122139 9984 raft_consensus.cc:348] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432bada5074140bf97a004319ab9fff8" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35861 } }
I20260809 13:01:18.122592 9984 raft_consensus.cc:374] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:18.122720 9984 raft_consensus.cc:729] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 432bada5074140bf97a004319ab9fff8, State: Initialized, Role: FOLLOWER
I20260809 13:01:18.123291 9984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432bada5074140bf97a004319ab9fff8" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35861 } }
I20260809 13:01:18.123544 9984 raft_consensus.cc:388] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:01:18.123684 9984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:01:18.123865 9984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:18.126410 9984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432bada5074140bf97a004319ab9fff8" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35861 } }
I20260809 13:01:18.127048 9984 leader_election.cc:302] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 432bada5074140bf97a004319ab9fff8; no voters:
I20260809 13:01:18.127651 9984 leader_election.cc:288] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:01:18.127816 9990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:18.128605 9990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8 [term 1 LEADER]: Becoming Leader. State: Replica: 432bada5074140bf97a004319ab9fff8, State: Running, Role: LEADER
I20260809 13:01:18.129163 9990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432bada5074140bf97a004319ab9fff8" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35861 } }
I20260809 13:01:18.129837 9984 sys_catalog.cc:564] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:01:18.131889 9991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "432bada5074140bf97a004319ab9fff8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432bada5074140bf97a004319ab9fff8" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35861 } } }
I20260809 13:01:18.132273 9991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:18.133124 9998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:01:18.133541 9992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 432bada5074140bf97a004319ab9fff8. Latest consensus state: current_term: 1 leader_uuid: "432bada5074140bf97a004319ab9fff8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432bada5074140bf97a004319ab9fff8" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 35861 } } }
I20260809 13:01:18.133829 9992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:18.142087 9998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:01:18.148866 10010 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 13:01:18.152510 9998 catalog_manager.cc:1347] Generated new cluster ID: 5c09db4fffb646c1a2c3be6cdb953766
I20260809 13:01:18.152616 9998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:01:18.171563 9998 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:01:18.173039 9998 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:01:18.182117 9998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8: Generated new TSK 0
I20260809 13:01:18.183272 9998 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:01:18.188768 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:35861
--builtin_ntp_servers=127.21.235.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 13:01:18.552652 10013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:18.553037 10013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:18.553139 10013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:18.560446 10013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:18.560547 10013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:18.560654 10013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 13:01:18.569707 10013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:35861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:18.570950 10013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:18.572638 10013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:18.586112 10022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:18.586365 10024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:18.586865 10020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:18.587036 10013 server_base.cc:1034] running on GCE node
I20260809 13:01:18.587674 10013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:18.588620 10013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:18.589968 10013 hybrid_clock.cc:648] HybridClock initialized: now 1786280478589914 us; error 60 us; skew 500 ppm
I20260809 13:01:18.590375 10013 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:01:18.593248 10013 webserver.cc:460] Webserver started at http://127.21.235.65:33519/ using document root <none> and password file <none>
I20260809 13:01:18.594003 10013 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:18.594156 10013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:18.594578 10013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:18.597692 10013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "3570e932c8a946ea8fb140bb62ca8e96"
format_stamp: "Formatted at 2026-08-09 13:01:18 on dist-test-slave-gp6n"
server_key: "f657ac478d487fd06ff21f6d0a3786cd"
server_key_iv: "6f147b2f51c7c0285cf31ce2c431104a"
server_key_version: "encryptionkey@0"
I20260809 13:01:18.598641 10013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "3570e932c8a946ea8fb140bb62ca8e96"
format_stamp: "Formatted at 2026-08-09 13:01:18 on dist-test-slave-gp6n"
server_key: "f657ac478d487fd06ff21f6d0a3786cd"
server_key_iv: "6f147b2f51c7c0285cf31ce2c431104a"
server_key_version: "encryptionkey@0"
I20260809 13:01:18.604079 10013 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 13:01:18.607755 10030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:18.609043 10013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:01:18.609222 10013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "3570e932c8a946ea8fb140bb62ca8e96"
format_stamp: "Formatted at 2026-08-09 13:01:18 on dist-test-slave-gp6n"
server_key: "f657ac478d487fd06ff21f6d0a3786cd"
server_key_iv: "6f147b2f51c7c0285cf31ce2c431104a"
server_key_version: "encryptionkey@0"
I20260809 13:01:18.609455 10013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:18.632006 10013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:18.632777 10013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:18.633046 10013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:18.633994 10013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:01:18.635638 10013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:01:18.635726 10013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:18.635807 10013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:01:18.635864 10013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:18.670035 10013 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:44213
I20260809 13:01:18.670152 10155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:44213 every 8 connection(s)
I20260809 13:01:18.671402 10013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 13:01:18.681175 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 10013
I20260809 13:01:18.681388 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 13:01:18.682003 22445 external_mini_cluster.cc:1428] Setting key dc7d866da76255fa45d83547201dace7
I20260809 13:01:18.684424 10158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:18.673999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:45287 (local address 127.21.235.126:35861)
0809 13:01:18.674550 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:18.674554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:18.675585 (+ 1031us) server_negotiation.cc:407] Connection header received
0809 13:01:18.677463 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:18.677478 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:18.677577 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:18.677705 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:18.678870 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:18.679618 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:18.681300 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:18.681826 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:18.681967 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:18.683516 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:18.683522 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:18.683525 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:18.683544 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:18.683586 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:18.683599 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:18.683679 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:18.683973 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 13:01:18.684071 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":262,"threads_started":1}
I20260809 13:01:18.685112 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:35861
--builtin_ntp_servers=127.21.235.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 13:01:18.685606 10159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:18.674490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:35861 (local address 127.21.235.65:45287)
0809 13:01:18.675414 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:18.675450 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:18.677171 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:18.677952 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:18.677961 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:18.678057 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:01:18.678690 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:18.678709 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:18.679788 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:18.679793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:18.681133 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:18.681152 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:18.682076 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:18.682085 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:18.682335 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:18.683152 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:01:18.683156 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:01:18.683211 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:01:18.683322 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:01:18.683711 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:01:18.683722 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:01:18.683731 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 13:01:18.683943 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 13:01:18.684358 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":280,"threads_started":1}
I20260809 13:01:18.687280 10156 heartbeater.cc:347] Connected to a master server at 127.21.235.126:35861
I20260809 13:01:18.687611 10156 heartbeater.cc:464] Registering TS with master...
I20260809 13:01:18.688365 10156 heartbeater.cc:511] Master 127.21.235.126:35861 requested a full tablet report, sending...
I20260809 13:01:18.690369 9946 ts_manager.cc:194] Registered new tserver with Master: 3570e932c8a946ea8fb140bb62ca8e96 (127.21.235.65:44213)
I20260809 13:01:18.691840 9946 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.235.65:45287
W20260809 13:01:19.025898 10161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:19.026181 10161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:19.026269 10161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:19.033661 10161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:19.033774 10161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:19.033882 10161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 13:01:19.043905 10161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:35861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:19.045373 10161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:19.047256 10161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:19.060904 10167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:19.060932 10170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:19.060992 10168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:19.061693 10161 server_base.cc:1034] running on GCE node
I20260809 13:01:19.062288 10161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:19.063297 10161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:19.064507 10161 hybrid_clock.cc:648] HybridClock initialized: now 1786280479064465 us; error 32 us; skew 500 ppm
I20260809 13:01:19.064802 10161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:01:19.067083 10161 webserver.cc:460] Webserver started at http://127.21.235.66:44773/ using document root <none> and password file <none>
I20260809 13:01:19.067781 10161 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:19.067940 10161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:19.068356 10161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:19.071385 10161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "8d796afb9a84400dbbdaf4bef0ad21a2"
format_stamp: "Formatted at 2026-08-09 13:01:19 on dist-test-slave-gp6n"
server_key: "93ee916e329e775ac3afc566fa41dc44"
server_key_iv: "f3d612c058dfc9c2fe77181668818d76"
server_key_version: "encryptionkey@0"
I20260809 13:01:19.072206 10161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "8d796afb9a84400dbbdaf4bef0ad21a2"
format_stamp: "Formatted at 2026-08-09 13:01:19 on dist-test-slave-gp6n"
server_key: "93ee916e329e775ac3afc566fa41dc44"
server_key_iv: "f3d612c058dfc9c2fe77181668818d76"
server_key_version: "encryptionkey@0"
I20260809 13:01:19.077278 10161 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:01:19.080894 10177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:19.082231 10161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 13:01:19.082456 10161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "8d796afb9a84400dbbdaf4bef0ad21a2"
format_stamp: "Formatted at 2026-08-09 13:01:19 on dist-test-slave-gp6n"
server_key: "93ee916e329e775ac3afc566fa41dc44"
server_key_iv: "f3d612c058dfc9c2fe77181668818d76"
server_key_version: "encryptionkey@0"
I20260809 13:01:19.082690 10161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:19.110647 10161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:19.111526 10161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:19.111832 10161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:19.112877 10161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:01:19.114526 10161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:01:19.114645 10161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:19.114782 10161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:01:19.114871 10161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:19.150367 10161 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:37533
I20260809 13:01:19.150494 10304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:37533 every 8 connection(s)
I20260809 13:01:19.151741 10161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 13:01:19.156683 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 10161
I20260809 13:01:19.156885 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 13:01:19.157483 22445 external_mini_cluster.cc:1428] Setting key b9c4bb4418b45d70e985ef4cd06bf66e
I20260809 13:01:19.160756 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:35861
--builtin_ntp_servers=127.21.235.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 13:01:19.166707 10158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:19.154599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:57389 (local address 127.21.235.126:35861)
0809 13:01:19.154794 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:19.154799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:19.156150 (+ 1351us) server_negotiation.cc:407] Connection header received
0809 13:01:19.158770 (+ 2620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:19.158776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:19.158853 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:19.158963 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:19.160345 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.161195 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.163795 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.164183 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.164282 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:19.165918 (+ 1636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:19.165923 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:19.165926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:19.165945 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:19.165977 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:19.165989 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:19.166064 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:19.166324 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 13:01:19.166451 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":69}
I20260809 13:01:19.168264 10307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:19.155100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:35861 (local address 127.21.235.66:57389)
0809 13:01:19.155985 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:19.156028 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:19.158460 (+ 2432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:19.159172 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:19.159180 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:19.159267 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:01:19.160141 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:19.160167 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.161369 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.161376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:19.163632 (+ 2256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:19.163650 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.164425 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.164434 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:19.164761 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:19.165544 (+ 783us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:01:19.165549 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:01:19.165606 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:01:19.165712 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:01:19.166093 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:01:19.166105 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:01:19.166116 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 13:01:19.166305 (+ 189us) client_negotiation.cc:240] Negotiation successful
0809 13:01:19.166804 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":276,"threads_started":1}
I20260809 13:01:19.170049 10305 heartbeater.cc:347] Connected to a master server at 127.21.235.126:35861
I20260809 13:01:19.170384 10305 heartbeater.cc:464] Registering TS with master...
I20260809 13:01:19.171330 10305 heartbeater.cc:511] Master 127.21.235.126:35861 requested a full tablet report, sending...
I20260809 13:01:19.173000 9946 ts_manager.cc:194] Registered new tserver with Master: 8d796afb9a84400dbbdaf4bef0ad21a2 (127.21.235.66:37533)
I20260809 13:01:19.173885 9946 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.235.66:57389
W20260809 13:01:19.504786 10308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:19.505158 10308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:19.505342 10308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:19.512889 10308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:19.513053 10308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:19.513242 10308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 13:01:19.522715 10308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:35861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:19.524111 10308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:19.525882 10308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:19.539263 10313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:19.539268 10314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:19.539486 10316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:19.540236 10308 server_base.cc:1034] running on GCE node
I20260809 13:01:19.540854 10308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:19.541847 10308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:19.543118 10308 hybrid_clock.cc:648] HybridClock initialized: now 1786280479543058 us; error 61 us; skew 500 ppm
I20260809 13:01:19.543437 10308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:01:19.545660 10308 webserver.cc:460] Webserver started at http://127.21.235.67:36873/ using document root <none> and password file <none>
I20260809 13:01:19.546329 10308 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:19.546492 10308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:19.546803 10308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:19.549652 10308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "16458ad08b1f4fb3b5a2ed4a24bc5152"
format_stamp: "Formatted at 2026-08-09 13:01:19 on dist-test-slave-gp6n"
server_key: "c6af2c07d6e47d62ce788064101fc656"
server_key_iv: "1df3801ba1efc6e84a42c36bc0459802"
server_key_version: "encryptionkey@0"
I20260809 13:01:19.550537 10308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "16458ad08b1f4fb3b5a2ed4a24bc5152"
format_stamp: "Formatted at 2026-08-09 13:01:19 on dist-test-slave-gp6n"
server_key: "c6af2c07d6e47d62ce788064101fc656"
server_key_iv: "1df3801ba1efc6e84a42c36bc0459802"
server_key_version: "encryptionkey@0"
I20260809 13:01:19.555738 10308 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 13:01:19.559134 10322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:19.560428 10308 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:01:19.560609 10308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "16458ad08b1f4fb3b5a2ed4a24bc5152"
format_stamp: "Formatted at 2026-08-09 13:01:19 on dist-test-slave-gp6n"
server_key: "c6af2c07d6e47d62ce788064101fc656"
server_key_iv: "1df3801ba1efc6e84a42c36bc0459802"
server_key_version: "encryptionkey@0"
I20260809 13:01:19.560827 10308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:19.601681 10308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:19.602646 10308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:19.602964 10308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:19.603951 10308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:01:19.605432 10308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:01:19.605563 10308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:19.605688 10308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:01:19.605777 10308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:19.639931 10308 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:41819
I20260809 13:01:19.639990 10449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:41819 every 8 connection(s)
I20260809 13:01:19.641252 10308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 13:01:19.643474 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 10308
I20260809 13:01:19.643713 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 13:01:19.644311 22445 external_mini_cluster.cc:1428] Setting key ec85062dfcce5748e452aa4e3a35ec7c
I20260809 13:01:19.654182 10158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:19.644208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:55127 (local address 127.21.235.126:35861)
0809 13:01:19.644433 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:19.644439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:19.646120 (+ 1681us) server_negotiation.cc:407] Connection header received
0809 13:01:19.648104 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:19.648108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:19.648172 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:19.648257 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:19.649230 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.650011 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.651616 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.652020 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.652117 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:19.653362 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:19.653367 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:19.653370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:19.653386 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:19.653425 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:19.653438 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:19.653507 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:19.653846 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 13:01:19.653929 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 13:01:19.655181 10452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:19.644817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:35861 (local address 127.21.235.67:55127)
0809 13:01:19.645911 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:19.645949 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:19.647858 (+ 1909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:19.648415 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:19.648424 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:19.648516 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:01:19.649082 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:19.649098 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.650145 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.650149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:19.651481 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:19.651493 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.652212 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.652217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:19.652416 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:19.653067 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:01:19.653069 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:01:19.653108 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:01:19.653187 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:01:19.653559 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:01:19.653571 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:01:19.653580 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 13:01:19.653765 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 13:01:19.654065 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":467,"threads_started":1}
I20260809 13:01:19.656612 10450 heartbeater.cc:347] Connected to a master server at 127.21.235.126:35861
I20260809 13:01:19.656922 10450 heartbeater.cc:464] Registering TS with master...
I20260809 13:01:19.657655 10450 heartbeater.cc:511] Master 127.21.235.126:35861 requested a full tablet report, sending...
I20260809 13:01:19.659354 9946 ts_manager.cc:194] Registered new tserver with Master: 16458ad08b1f4fb3b5a2ed4a24bc5152 (127.21.235.67:41819)
I20260809 13:01:19.660064 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:01:19.660346 9946 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.235.67:55127
I20260809 13:01:19.671025 10158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:19.664737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40520 (local address 127.21.235.126:35861)
0809 13:01:19.664925 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:19.664930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:19.665446 (+ 516us) server_negotiation.cc:407] Connection header received
0809 13:01:19.665601 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:19.665604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:19.665710 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:19.665808 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:19.666270 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.667059 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.668429 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.668955 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.669089 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:19.670244 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:19.670249 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:19.670253 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:19.670270 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:19.670301 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:19.670314 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:19.670387 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:19.670619 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 13:01:19.670700 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 13:01:19.675053 9946 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40520:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:01:19.681829 9946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:01:19.695744 10156 heartbeater.cc:503] Master 127.21.235.126:35861 was elected leader, sending a full tablet report...
I20260809 13:01:19.716643 10467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:19.695308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44368 (local address 127.21.235.67:41819)
0809 13:01:19.707974 (+ 12666us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:19.707984 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:19.708024 (+ 40us) server_negotiation.cc:407] Connection header received
0809 13:01:19.708153 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:19.708162 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:19.708391 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:19.708622 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:19.709435 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.711184 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.714325 (+ 3141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.715427 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.715589 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:19.715880 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:19.716099 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 13:01:19.716260 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12420,"thread_start_us":12256,"threads_started":1}
I20260809 13:01:19.716648 10465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:19.694610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:41819 (local address 127.0.0.1:44368)
0809 13:01:19.704366 (+ 9756us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:19.704424 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:19.704654 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:19.708618 (+ 3964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:19.708623 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:19.708655 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:19.709150 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:19.709167 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.711353 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.711359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:19.714075 (+ 2716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:19.714090 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.715624 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.715630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:19.715822 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:19.715882 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 13:01:19.716050 (+ 168us) client_negotiation.cc:240] Negotiation successful
0809 13:01:19.716244 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9382,"mutex_wait_us":129,"thread_start_us":12162,"threads_started":1}
I20260809 13:01:19.717865 10464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:19.695830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:44213 (local address 127.0.0.1:46772)
0809 13:01:19.707113 (+ 11283us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:19.707134 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:19.707258 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:19.708130 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:19.708138 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:19.708173 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:19.708791 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:19.708808 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.711442 (+ 2634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.711447 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:19.714509 (+ 3062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:19.714530 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.717081 (+ 2551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.717086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:19.717277 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:19.717311 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 13:01:19.717416 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 13:01:19.717533 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11143,"mutex_wait_us":48,"thread_start_us":9656,"threads_started":1}
I20260809 13:01:19.718225 10469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:19.696196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46772 (local address 127.21.235.65:44213)
0809 13:01:19.706239 (+ 10043us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:19.706252 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:19.707406 (+ 1154us) server_negotiation.cc:407] Connection header received
0809 13:01:19.707510 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:19.707518 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:19.707835 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:19.708961 (+ 1126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:19.709238 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.711207 (+ 1969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.714777 (+ 3570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.716406 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.716551 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:19.717092 (+ 541us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:19.717494 (+ 402us) server_negotiation.cc:299] Negotiation successful
0809 13:01:19.717692 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9810,"thread_start_us":9674,"threads_started":1}
I20260809 13:01:19.718611 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:19.695297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40822 (local address 127.21.235.66:37533)
0809 13:01:19.708294 (+ 12997us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:19.708303 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:19.708335 (+ 32us) server_negotiation.cc:407] Connection header received
0809 13:01:19.708565 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:19.708575 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:19.708801 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:19.709014 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:19.709965 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.711424 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.714786 (+ 3362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.716410 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.716551 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:19.716791 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:19.717968 (+ 1177us) server_negotiation.cc:299] Negotiation successful
0809 13:01:19.718152 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12791,"thread_start_us":12641,"threads_started":1}
I20260809 13:01:19.719062 10468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:19.694846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:37533 (local address 127.0.0.1:40822)
0809 13:01:19.705849 (+ 11003us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:19.705870 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:19.705992 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:19.709039 (+ 3047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:19.709043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:19.709313 (+ 270us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:19.709744 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:19.709757 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.711587 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.711595 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:19.714557 (+ 2962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:19.714573 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.717264 (+ 2691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.717272 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:19.717473 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:19.717509 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 13:01:19.717603 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 13:01:19.717734 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10826,"mutex_wait_us":171,"thread_start_us":8996,"threads_started":1}
I20260809 13:01:19.720935 10379 tablet_service.cc:1467] Processing CreateTablet for tablet 88bcdc0ffaae43289bcc38fbde9bca1b (DEFAULT_TABLE table=test-table [id=5073a1fc83534e0cbd30260273dad7b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:01:19.721421 10083 tablet_service.cc:1467] Processing CreateTablet for tablet 88bcdc0ffaae43289bcc38fbde9bca1b (DEFAULT_TABLE table=test-table [id=5073a1fc83534e0cbd30260273dad7b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:01:19.722388 10379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88bcdc0ffaae43289bcc38fbde9bca1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:19.722864 10083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88bcdc0ffaae43289bcc38fbde9bca1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:19.725219 10232 tablet_service.cc:1467] Processing CreateTablet for tablet 88bcdc0ffaae43289bcc38fbde9bca1b (DEFAULT_TABLE table=test-table [id=5073a1fc83534e0cbd30260273dad7b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:01:19.726639 10232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88bcdc0ffaae43289bcc38fbde9bca1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:19.734318 10470 tablet_bootstrap.cc:492] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152: Bootstrap starting.
I20260809 13:01:19.737609 10470 tablet_bootstrap.cc:654] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:19.738899 10471 tablet_bootstrap.cc:492] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96: Bootstrap starting.
I20260809 13:01:19.738871 10470 log.cc:824] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:19.742865 10473 tablet_bootstrap.cc:492] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2: Bootstrap starting.
I20260809 13:01:19.743975 10471 tablet_bootstrap.cc:654] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:19.745143 10471 log.cc:824] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:19.746378 10470 tablet_bootstrap.cc:492] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152: No bootstrap required, opened a new log
I20260809 13:01:19.746469 10473 tablet_bootstrap.cc:654] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:19.746742 10470 ts_tablet_manager.cc:1397] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.007s
I20260809 13:01:19.747453 10473 log.cc:824] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:19.748304 10471 tablet_bootstrap.cc:492] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96: No bootstrap required, opened a new log
I20260809 13:01:19.748574 10471 ts_tablet_manager.cc:1397] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260809 13:01:19.749902 10473 tablet_bootstrap.cc:492] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2: No bootstrap required, opened a new log
I20260809 13:01:19.750149 10473 ts_tablet_manager.cc:1397] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260809 13:01:19.757771 10473 raft_consensus.cc:348] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16458ad08b1f4fb3b5a2ed4a24bc5152" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41819 } } peers { permanent_uuid: "8d796afb9a84400dbbdaf4bef0ad21a2" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37533 } } peers { permanent_uuid: "3570e932c8a946ea8fb140bb62ca8e96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 44213 } }
I20260809 13:01:19.757696 10470 raft_consensus.cc:348] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16458ad08b1f4fb3b5a2ed4a24bc5152" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41819 } } peers { permanent_uuid: "8d796afb9a84400dbbdaf4bef0ad21a2" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37533 } } peers { permanent_uuid: "3570e932c8a946ea8fb140bb62ca8e96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 44213 } }
I20260809 13:01:19.758163 10473 raft_consensus.cc:374] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:19.758173 10470 raft_consensus.cc:374] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:19.758265 10470 raft_consensus.cc:729] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16458ad08b1f4fb3b5a2ed4a24bc5152, State: Initialized, Role: FOLLOWER
I20260809 13:01:19.758262 10473 raft_consensus.cc:729] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d796afb9a84400dbbdaf4bef0ad21a2, State: Initialized, Role: FOLLOWER
I20260809 13:01:19.758934 10473 consensus_queue.cc:260] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16458ad08b1f4fb3b5a2ed4a24bc5152" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41819 } } peers { permanent_uuid: "8d796afb9a84400dbbdaf4bef0ad21a2" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37533 } } peers { permanent_uuid: "3570e932c8a946ea8fb140bb62ca8e96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 44213 } }
I20260809 13:01:19.758952 10470 consensus_queue.cc:260] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16458ad08b1f4fb3b5a2ed4a24bc5152" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41819 } } peers { permanent_uuid: "8d796afb9a84400dbbdaf4bef0ad21a2" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37533 } } peers { permanent_uuid: "3570e932c8a946ea8fb140bb62ca8e96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 44213 } }
I20260809 13:01:19.759191 10471 raft_consensus.cc:348] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16458ad08b1f4fb3b5a2ed4a24bc5152" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41819 } } peers { permanent_uuid: "8d796afb9a84400dbbdaf4bef0ad21a2" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37533 } } peers { permanent_uuid: "3570e932c8a946ea8fb140bb62ca8e96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 44213 } }
I20260809 13:01:19.759706 10471 raft_consensus.cc:374] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:19.759840 10471 raft_consensus.cc:729] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3570e932c8a946ea8fb140bb62ca8e96, State: Initialized, Role: FOLLOWER
I20260809 13:01:19.760215 10305 heartbeater.cc:503] Master 127.21.235.126:35861 was elected leader, sending a full tablet report...
I20260809 13:01:19.760792 10473 ts_tablet_manager.cc:1428] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260809 13:01:19.760655 10471 consensus_queue.cc:260] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16458ad08b1f4fb3b5a2ed4a24bc5152" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41819 } } peers { permanent_uuid: "8d796afb9a84400dbbdaf4bef0ad21a2" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37533 } } peers { permanent_uuid: "3570e932c8a946ea8fb140bb62ca8e96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 44213 } }
I20260809 13:01:19.762691 10470 ts_tablet_manager.cc:1428] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152: Time spent starting tablet: real 0.016s user 0.014s sys 0.002s
I20260809 13:01:19.762828 10471 ts_tablet_manager.cc:1428] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96: Time spent starting tablet: real 0.014s user 0.013s sys 0.002s
I20260809 13:01:19.763487 10450 heartbeater.cc:503] Master 127.21.235.126:35861 was elected leader, sending a full tablet report...
I20260809 13:01:19.830546 10477 raft_consensus.cc:482] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:01:19.830956 10477 raft_consensus.cc:504] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16458ad08b1f4fb3b5a2ed4a24bc5152" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41819 } } peers { permanent_uuid: "8d796afb9a84400dbbdaf4bef0ad21a2" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37533 } } peers { permanent_uuid: "3570e932c8a946ea8fb140bb62ca8e96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 44213 } }
I20260809 13:01:19.832739 10477 leader_election.cc:288] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16458ad08b1f4fb3b5a2ed4a24bc5152 (127.21.235.67:41819), 3570e932c8a946ea8fb140bb62ca8e96 (127.21.235.65:44213)
I20260809 13:01:19.840189 10469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:19.833524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:44213 (local address 127.21.235.65:44213)
0809 13:01:19.833759 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:19.833766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:19.834249 (+ 483us) server_negotiation.cc:407] Connection header received
0809 13:01:19.834574 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:19.834578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:19.834655 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:19.834760 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:19.835567 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.836379 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.838274 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.839191 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.839423 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:19.839503 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:19.839779 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 13:01:19.839907 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 13:01:19.840209 10467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:19.833168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:32921 (local address 127.21.235.67:41819)
0809 13:01:19.833450 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:19.833460 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:19.834329 (+ 869us) server_negotiation.cc:407] Connection header received
0809 13:01:19.834612 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:19.834616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:19.834676 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:19.834772 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:19.835561 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.836259 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.838184 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.839281 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.839501 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:19.839571 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:19.839837 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 13:01:19.839964 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 13:01:19.840306 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:19.833087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:44213 (local address 127.21.235.66:44213)
0809 13:01:19.834111 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:19.834132 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:19.834371 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:19.834775 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:19.834779 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:19.834804 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:19.835416 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:19.835428 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.836515 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.836521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:19.838048 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:19.838060 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.839423 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.839431 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:19.839615 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:19.839639 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 13:01:19.839749 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 13:01:19.839912 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"mutex_wait_us":42,"thread_start_us":872,"threads_started":1}
I20260809 13:01:19.840718 10480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:19.833112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:41819 (local address 127.21.235.66:32921)
0809 13:01:19.834232 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:19.834246 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:19.834381 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:19.834786 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:19.834791 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:19.834813 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:19.835403 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:19.835414 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.836412 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.836418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:19.837992 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:19.838004 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.839515 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.839521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:19.839682 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:19.839705 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 13:01:19.839810 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 13:01:19.839943 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1025,"mutex_wait_us":48,"thread_start_us":544,"threads_started":1}
I20260809 13:01:19.841341 10107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88bcdc0ffaae43289bcc38fbde9bca1b" candidate_uuid: "8d796afb9a84400dbbdaf4bef0ad21a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3570e932c8a946ea8fb140bb62ca8e96" is_pre_election: true
I20260809 13:01:19.841945 10107 raft_consensus.cc:2454] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d796afb9a84400dbbdaf4bef0ad21a2 in term 0.
I20260809 13:01:19.842005 10400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88bcdc0ffaae43289bcc38fbde9bca1b" candidate_uuid: "8d796afb9a84400dbbdaf4bef0ad21a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16458ad08b1f4fb3b5a2ed4a24bc5152" is_pre_election: true
I20260809 13:01:19.842840 10180 leader_election.cc:302] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3570e932c8a946ea8fb140bb62ca8e96, 8d796afb9a84400dbbdaf4bef0ad21a2; no voters:
I20260809 13:01:19.842931 10400 raft_consensus.cc:2454] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d796afb9a84400dbbdaf4bef0ad21a2 in term 0.
I20260809 13:01:19.843257 10477 raft_consensus.cc:2781] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:01:19.843433 10477 raft_consensus.cc:482] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:01:19.843575 10477 raft_consensus.cc:3037] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:19.845978 10477 raft_consensus.cc:504] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16458ad08b1f4fb3b5a2ed4a24bc5152" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41819 } } peers { permanent_uuid: "8d796afb9a84400dbbdaf4bef0ad21a2" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37533 } } peers { permanent_uuid: "3570e932c8a946ea8fb140bb62ca8e96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 44213 } }
I20260809 13:01:19.846649 10477 leader_election.cc:288] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [CANDIDATE]: Term 1 election: Requested vote from peers 16458ad08b1f4fb3b5a2ed4a24bc5152 (127.21.235.67:41819), 3570e932c8a946ea8fb140bb62ca8e96 (127.21.235.65:44213)
I20260809 13:01:19.847129 10400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88bcdc0ffaae43289bcc38fbde9bca1b" candidate_uuid: "8d796afb9a84400dbbdaf4bef0ad21a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16458ad08b1f4fb3b5a2ed4a24bc5152"
I20260809 13:01:19.847162 10107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88bcdc0ffaae43289bcc38fbde9bca1b" candidate_uuid: "8d796afb9a84400dbbdaf4bef0ad21a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3570e932c8a946ea8fb140bb62ca8e96"
I20260809 13:01:19.847339 10400 raft_consensus.cc:3037] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:19.847496 10107 raft_consensus.cc:3037] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:19.849756 10400 raft_consensus.cc:2454] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d796afb9a84400dbbdaf4bef0ad21a2 in term 1.
I20260809 13:01:19.849915 10107 raft_consensus.cc:2454] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d796afb9a84400dbbdaf4bef0ad21a2 in term 1.
I20260809 13:01:19.850172 10182 leader_election.cc:302] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16458ad08b1f4fb3b5a2ed4a24bc5152, 8d796afb9a84400dbbdaf4bef0ad21a2; no voters:
I20260809 13:01:19.850554 10477 raft_consensus.cc:2781] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:19.851007 10477 raft_consensus.cc:686] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [term 1 LEADER]: Becoming Leader. State: Replica: 8d796afb9a84400dbbdaf4bef0ad21a2, State: Running, Role: LEADER
I20260809 13:01:19.851468 10477 consensus_queue.cc:237] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16458ad08b1f4fb3b5a2ed4a24bc5152" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41819 } } peers { permanent_uuid: "8d796afb9a84400dbbdaf4bef0ad21a2" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37533 } } peers { permanent_uuid: "3570e932c8a946ea8fb140bb62ca8e96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 44213 } }
I20260809 13:01:19.856247 9946 catalog_manager.cc:5495] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d796afb9a84400dbbdaf4bef0ad21a2 (127.21.235.66). New cstate: current_term: 1 leader_uuid: "8d796afb9a84400dbbdaf4bef0ad21a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16458ad08b1f4fb3b5a2ed4a24bc5152" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d796afb9a84400dbbdaf4bef0ad21a2" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3570e932c8a946ea8fb140bb62ca8e96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 44213 } health_report { overall_health: UNKNOWN } } }
W20260809 13:01:19.893039 10451 tablet.cc:2369] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:01:19.903971 10306 tablet.cc:2369] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:01:19.905462 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:19.899830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40838 (local address 127.21.235.66:37533)
0809 13:01:19.900050 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:19.900058 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:19.900125 (+ 67us) server_negotiation.cc:407] Connection header received
0809 13:01:19.900384 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:19.900388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:19.900492 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:19.900655 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:01:19.901029 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.901749 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.903300 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.903724 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.903863 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:19.904346 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:01:19.904871 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:01:19.905054 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:19.905105 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 13:01:19.905189 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 13:01:19.914391 10107 raft_consensus.cc:1261] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96 [term 1 FOLLOWER]: Refusing update from remote peer 8d796afb9a84400dbbdaf4bef0ad21a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:01:19.914390 10400 raft_consensus.cc:1261] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152 [term 1 FOLLOWER]: Refusing update from remote peer 8d796afb9a84400dbbdaf4bef0ad21a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:01:19.915620 10482 consensus_queue.cc:1035] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16458ad08b1f4fb3b5a2ed4a24bc5152" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 41819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:01:19.915968 10477 consensus_queue.cc:1035] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3570e932c8a946ea8fb140bb62ca8e96" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 44213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 13:01:19.923815 10157 tablet.cc:2369] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:01:19.928690 10485 mvcc.cc:205] Tried to move back new op lower bound from 7316604845717848064 to 7316604845475770368. Current Snapshot: MvccSnapshot[applied={T|T < 7316604845717848064}]
I20260809 13:01:19.946523 9946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:01:19.946842 9946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:01:19.950528 9946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 432bada5074140bf97a004319ab9fff8: Sending DeleteTablet for 3 replicas of tablet 88bcdc0ffaae43289bcc38fbde9bca1b
I20260809 13:01:19.951779 10232 tablet_service.cc:1514] Processing DeleteTablet for tablet 88bcdc0ffaae43289bcc38fbde9bca1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:01:19 UTC) from {username='slave'} at 127.0.0.1:40822
I20260809 13:01:19.951779 10379 tablet_service.cc:1514] Processing DeleteTablet for tablet 88bcdc0ffaae43289bcc38fbde9bca1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:01:19 UTC) from {username='slave'} at 127.0.0.1:44368
I20260809 13:01:19.952071 10083 tablet_service.cc:1514] Processing DeleteTablet for tablet 88bcdc0ffaae43289bcc38fbde9bca1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:01:19 UTC) from {username='slave'} at 127.0.0.1:46772
I20260809 13:01:19.954280 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 10013
I20260809 13:01:19.964133 10499 tablet_replica.cc:330] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2: stopping tablet replica
I20260809 13:01:19.965866 10499 raft_consensus.cc:2229] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:01:19.966692 10499 raft_consensus.cc:2258] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:01:19.970481 10499 ts_tablet_manager.cc:1905] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:01:19.975515 10500 tablet_replica.cc:330] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152: stopping tablet replica
I20260809 13:01:19.975986 10500 raft_consensus.cc:2229] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:01:19.976576 10500 raft_consensus.cc:2258] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:01:19.978037 10502 tablet_replica.cc:330] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96: stopping tablet replica
I20260809 13:01:19.978554 10502 raft_consensus.cc:2229] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:01:19.979024 10500 ts_tablet_manager.cc:1905] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:01:19.981132 10502 raft_consensus.cc:2258] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:01:19.981372 10499 ts_tablet_manager.cc:1918] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:01:19.981570 10499 log.cc:1196] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/88bcdc0ffaae43289bcc38fbde9bca1b
I20260809 13:01:19.982209 10499 ts_tablet_manager.cc:1939] T 88bcdc0ffaae43289bcc38fbde9bca1b P 8d796afb9a84400dbbdaf4bef0ad21a2: Deleting consensus metadata
I20260809 13:01:19.984354 9931 catalog_manager.cc:4841] TS 8d796afb9a84400dbbdaf4bef0ad21a2 (127.21.235.66:37533): tablet 88bcdc0ffaae43289bcc38fbde9bca1b (table test-table [id=5073a1fc83534e0cbd30260273dad7b8]) successfully deleted
I20260809 13:01:19.985380 10502 ts_tablet_manager.cc:1905] T 88bcdc0ffaae43289bcc38fbde9bca1b P 3570e932c8a946ea8fb140bb62ca8e96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:01:19.986047 10500 ts_tablet_manager.cc:1918] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:01:19.986208 10500 log.cc:1196] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/88bcdc0ffaae43289bcc38fbde9bca1b
I20260809 13:01:19.986785 10500 ts_tablet_manager.cc:1939] T 88bcdc0ffaae43289bcc38fbde9bca1b P 16458ad08b1f4fb3b5a2ed4a24bc5152: Deleting consensus metadata
I20260809 13:01:19.986848 10469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:19.957899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46788 (local address 127.21.235.65:44213)
0809 13:01:19.958141 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:19.958150 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:19.971349 (+ 13199us) server_negotiation.cc:407] Connection header received
0809 13:01:19.971727 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:19.971737 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:19.971885 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:19.971984 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:19.972938 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.974589 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.978795 (+ 4206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:19.979467 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:19.979571 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:19.984867 (+ 5296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:19.984880 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:19.984899 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:19.985740 (+ 841us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:19.985819 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:19.985846 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:19.985946 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:19.986262 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 13:01:19.986442 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 13:01:19.988137 10150 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:19.988541 9932 catalog_manager.cc:4841] TS 16458ad08b1f4fb3b5a2ed4a24bc5152 (127.21.235.67:41819): tablet 88bcdc0ffaae43289bcc38fbde9bca1b (table test-table [id=5073a1fc83534e0cbd30260273dad7b8]) successfully deleted
I20260809 13:01:20.086112 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 10013
W20260809 13:01:20.118533 9929 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.235.65:44213 (error 108)
I20260809 13:01:20.119364 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 10161
I20260809 13:01:20.129031 10468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:20.128482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:44213 (local address 127.0.0.1:46796)
0809 13:01:20.128689 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:20.128844 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 13:01:20.129473 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:20.120941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40848 (local address 127.21.235.66:37533)
0809 13:01:20.121131 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:20.121137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:20.121157 (+ 20us) server_negotiation.cc:407] Connection header received
0809 13:01:20.121239 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:20.121244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:20.121365 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:20.121487 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:20.122181 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:20.123443 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:20.125420 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:20.126097 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:20.126228 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:20.127766 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:20.127776 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:20.127790 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:20.128549 (+ 759us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:20.128613 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:20.128641 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:20.128790 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:20.128920 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 13:01:20.129030 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260809 13:01:20.129456 9929 catalog_manager.cc:4584] TS 3570e932c8a946ea8fb140bb62ca8e96 (127.21.235.65:44213): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 88bcdc0ffaae43289bcc38fbde9bca1b: Network error: Client connection negotiation failed: client connection to 127.21.235.65:44213: connect: Connection refused (error 111)
I20260809 13:01:20.130308 10299 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:20.183359 10468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:20.182793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:44213 (local address 127.0.0.1:46810)
0809 13:01:20.183043 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:20.183157 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 13:01:20.184432 10468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:20.184045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:44213 (local address 127.0.0.1:46826)
0809 13:01:20.184200 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:20.184283 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 13:01:20.245080 10468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:20.244417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:44213 (local address 127.0.0.1:46840)
0809 13:01:20.244728 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:20.244856 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260809 13:01:20.246363 10468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:20.245895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:44213 (local address 127.0.0.1:46850)
0809 13:01:20.246095 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:20.246193 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 13:01:20.252170 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 10161
I20260809 13:01:20.276347 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 10308
I20260809 13:01:20.283716 10467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:20.277140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44372 (local address 127.21.235.67:41819)
0809 13:01:20.277348 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:20.277353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:20.277372 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:01:20.277531 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:20.277535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:20.277617 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:20.277706 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:20.278122 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:20.278935 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:20.280319 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:20.280702 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:20.280796 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:20.282072 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:20.282080 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:20.282095 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:20.282923 (+ 828us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:20.282970 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:20.282991 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:20.283087 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:20.283244 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 13:01:20.283370 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 13:01:20.284420 10444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:20.327405 10468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:20.326922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:44213 (local address 127.0.0.1:46860)
0809 13:01:20.327137 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:20.327229 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 13:01:20.328503 10468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:20.328084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:44213 (local address 127.0.0.1:46866)
0809 13:01:20.328287 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:20.328394 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 13:01:20.375245 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 10308
I20260809 13:01:20.395244 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 9912
I20260809 13:01:20.396389 9979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:20.507489 10468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:20.506923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:44213 (local address 127.0.0.1:46870)
0809 13:01:20.507149 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:20.507256 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 13:01:20.507622 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 9912
2026-08-09T13:01:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2966 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:01:20 dist-test-slave-gp6n krb5kdc[10512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:01:20 dist-test-slave-gp6n krb5kdc[10512](info): set up 2 sockets
Aug 09 13:01:20 dist-test-slave-gp6n krb5kdc[10512](info): commencing operation
krb5kdc: starting...
W20260809 13:01:21.700680 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:01:21 dist-test-slave-gp6n krb5kdc[10512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:01:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:01:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 13:01:22.007311 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:46491
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:42787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:46491
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:22.361761 10527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:22.362118 10527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:22.362288 10527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:22.369572 10527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:01:22.369716 10527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:22.369825 10527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:22.369932 10527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:01:22.370023 10527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:01:22.380236 10527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:42787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:46491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:46491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:22.381675 10527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:22.383102 10527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:22.395287 10537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:22.395396 10534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:22.395838 10535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:22.396395 10527 server_base.cc:1034] running on GCE node
I20260809 13:01:22.397172 10527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:22.398658 10527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:22.399888 10527 hybrid_clock.cc:648] HybridClock initialized: now 1786280482399845 us; error 51 us; skew 500 ppm
I20260809 13:01:22.400238 10527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:22 dist-test-slave-gp6n krb5kdc[10512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280482, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:22.403573 10527 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 13:01:22.405143 10527 webserver.cc:460] Webserver started at http://127.21.235.126:43999/ using document root <none> and password file <none>
I20260809 13:01:22.405836 10527 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:22.405987 10527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:22.406351 10527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:22.409482 10527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "49bda392e8214f8f8106f071c1a404bc"
format_stamp: "Formatted at 2026-08-09 13:01:22 on dist-test-slave-gp6n"
server_key: "d2c1b770fd06bd8907c2c941bbae46d5"
server_key_iv: "e6046ba25687a2329c6da10636a38b12"
server_key_version: "encryptionkey@0"
I20260809 13:01:22.410244 10527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "49bda392e8214f8f8106f071c1a404bc"
format_stamp: "Formatted at 2026-08-09 13:01:22 on dist-test-slave-gp6n"
server_key: "d2c1b770fd06bd8907c2c941bbae46d5"
server_key_iv: "e6046ba25687a2329c6da10636a38b12"
server_key_version: "encryptionkey@0"
I20260809 13:01:22.415704 10527 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 13:01:22.419387 10546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:22.420673 10527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 13:01:22.420866 10527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "49bda392e8214f8f8106f071c1a404bc"
format_stamp: "Formatted at 2026-08-09 13:01:22 on dist-test-slave-gp6n"
server_key: "d2c1b770fd06bd8907c2c941bbae46d5"
server_key_iv: "e6046ba25687a2329c6da10636a38b12"
server_key_version: "encryptionkey@0"
I20260809 13:01:22.421077 10527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:22.439951 10527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:22.444398 10527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:22.444747 10527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:22.464676 10527 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:46491
I20260809 13:01:22.464709 10609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:46491 every 8 connection(s)
I20260809 13:01:22.466109 10527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:01:22.467751 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 10527
I20260809 13:01:22.467947 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:01:22.468498 22445 external_mini_cluster.cc:1428] Setting key f8eb9d5ad72c97a32de8e36b91846cff
I20260809 13:01:22.471069 10611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 13:01:22 dist-test-slave-gp6n krb5kdc[10512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:01:22.485080 10611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc: Bootstrap starting.
I20260809 13:01:22.489454 10611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:22.490794 10611 log.cc:824] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:22.494561 10611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc: No bootstrap required, opened a new log
I20260809 13:01:22.494585 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:22.470516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53556 (local address 127.21.235.126:46491)
0809 13:01:22.471326 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:22.471338 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:22.471381 (+ 43us) server_negotiation.cc:407] Connection header received
0809 13:01:22.473039 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:22.473061 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:22.473465 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:22.473812 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:22.475815 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:22.477267 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:22.479286 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:22.479948 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:22.480084 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:22.485229 (+ 5145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:22.485241 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:22.485260 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:22.485310 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:22.489589 (+ 4279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:22.490301 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:22.490313 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:22.490323 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:22.490594 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:22.491027 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:22.491031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:22.491034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:22.491770 (+ 736us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:22.492023 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:22.492687 (+ 664us) server_negotiation.cc:299] Negotiation successful
0809 13:01:22.493129 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":376,"threads_started":1}
I20260809 13:01:22.502904 10611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49bda392e8214f8f8106f071c1a404bc" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46491 } }
I20260809 13:01:22.503322 10611 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:22.503405 10611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49bda392e8214f8f8106f071c1a404bc, State: Initialized, Role: FOLLOWER
I20260809 13:01:22.503973 10611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49bda392e8214f8f8106f071c1a404bc" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46491 } }
I20260809 13:01:22.504191 10611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:01:22.504294 10611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:01:22.504419 10611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:22.506608 10611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49bda392e8214f8f8106f071c1a404bc" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46491 } }
I20260809 13:01:22.507068 10611 leader_election.cc:302] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49bda392e8214f8f8106f071c1a404bc; no voters:
I20260809 13:01:22.507627 10611 leader_election.cc:288] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:01:22.507792 10616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:22.508885 10616 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc [term 1 LEADER]: Becoming Leader. State: Replica: 49bda392e8214f8f8106f071c1a404bc, State: Running, Role: LEADER
I20260809 13:01:22.509585 10616 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49bda392e8214f8f8106f071c1a404bc" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46491 } }
I20260809 13:01:22.510028 10611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:01:22.512919 10617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49bda392e8214f8f8106f071c1a404bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49bda392e8214f8f8106f071c1a404bc" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46491 } } }
I20260809 13:01:22.512892 10618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49bda392e8214f8f8106f071c1a404bc. Latest consensus state: current_term: 1 leader_uuid: "49bda392e8214f8f8106f071c1a404bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49bda392e8214f8f8106f071c1a404bc" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46491 } } }
I20260809 13:01:22.513432 10617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:22.513455 10618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:22.515666 10627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:01:22.520705 10627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:01:22.521425 10637 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 13:01:22.530100 10627 catalog_manager.cc:1347] Generated new cluster ID: 3af6e4624372456cb984e0874d26275d
I20260809 13:01:22.530263 10627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:01:22.539422 10627 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:01:22.541126 10627 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:01:22.547926 10627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc: Generated new TSK 0
I20260809 13:01:22.548758 10627 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
I20260809 13:01:22.622926 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:46491
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:42787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:22.998785 10642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:22.999075 10642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:22.999233 10642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:23.006887 10642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:23.007007 10642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:23.007213 10642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 13:01:23.017704 10642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:42787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:46491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:23.019094 10642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:23.020434 10642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:23.034150 10647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:23.034237 10648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:23.034193 10650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:23.035207 10642 server_base.cc:1034] running on GCE node
I20260809 13:01:23.035859 10642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:23.036803 10642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:23.038014 10642 hybrid_clock.cc:648] HybridClock initialized: now 1786280483037945 us; error 57 us; skew 500 ppm
I20260809 13:01:23.038347 10642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:23 dist-test-slave-gp6n krb5kdc[10512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280483, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:23.047343 10642 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 13:01:23.049080 10642 webserver.cc:460] Webserver started at http://127.21.235.65:39321/ using document root <none> and password file <none>
I20260809 13:01:23.049911 10642 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:23.050062 10642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:23.050395 10642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:23.053534 10642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "cc30572649ef418e9ed3e56ac4a6095f"
format_stamp: "Formatted at 2026-08-09 13:01:23 on dist-test-slave-gp6n"
server_key: "80b4fe2f7b7e3011694faee27fbee2c3"
server_key_iv: "2ea28ac720994ffe05eaab0c62cd067b"
server_key_version: "encryptionkey@0"
I20260809 13:01:23.054646 10642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "cc30572649ef418e9ed3e56ac4a6095f"
format_stamp: "Formatted at 2026-08-09 13:01:23 on dist-test-slave-gp6n"
server_key: "80b4fe2f7b7e3011694faee27fbee2c3"
server_key_iv: "2ea28ac720994ffe05eaab0c62cd067b"
server_key_version: "encryptionkey@0"
I20260809 13:01:23.060093 10642 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 13:01:23.063580 10658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:23.064921 10642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 13:01:23.065086 10642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "cc30572649ef418e9ed3e56ac4a6095f"
format_stamp: "Formatted at 2026-08-09 13:01:23 on dist-test-slave-gp6n"
server_key: "80b4fe2f7b7e3011694faee27fbee2c3"
server_key_iv: "2ea28ac720994ffe05eaab0c62cd067b"
server_key_version: "encryptionkey@0"
I20260809 13:01:23.065269 10642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:23.081188 10642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:23.085371 10642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:23.085677 10642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:23.086720 10642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:01:23.088145 10642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:01:23.088220 10642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:23.088404 10642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:01:23.088465 10642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:23.122661 10642 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:38825
I20260809 13:01:23.122701 10787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:38825 every 8 connection(s)
I20260809 13:01:23.124045 10642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 13:01:23.127717 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 10642
I20260809 13:01:23.127943 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 13:01:23.128413 22445 external_mini_cluster.cc:1428] Setting key aa9ed40551541a3b436584c85594c8e9
Aug 09 13:01:23 dist-test-slave-gp6n krb5kdc[10512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280483, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:01:23.147372 10791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:23.126943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:40021 (local address 127.21.235.126:46491)
0809 13:01:23.127471 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:23.127480 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:23.128587 (+ 1107us) server_negotiation.cc:407] Connection header received
0809 13:01:23.130783 (+ 2196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:23.130789 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:23.130883 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:23.131005 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:23.132546 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:23.133742 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:23.135444 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:23.136145 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:23.136299 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:23.140130 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:23.140140 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:23.140147 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:23.140194 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:23.143701 (+ 3507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:23.144762 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:23.144766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:23.144769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:23.144902 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:23.145468 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:23.145472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:23.145474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:23.145759 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:23.145885 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:23.146889 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0809 13:01:23.147085 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":280,"threads_started":1}
I20260809 13:01:23.149067 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:23.127415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:46491 (local address 127.21.235.65:40021)
0809 13:01:23.128364 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:23.128418 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:23.130485 (+ 2067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:23.131219 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:23.131228 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:23.131776 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:01:23.132360 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:23.132378 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:23.133942 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:23.133946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:23.135298 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:23.135309 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:23.136368 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:23.136375 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:23.136588 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:23.137353 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:01:23.137381 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:01:23.139873 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:01:23.143864 (+ 3991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:01:23.143877 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:01:23.143898 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:01:23.144623 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:01:23.145017 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:01:23.145025 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:01:23.145031 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:01:23.145298 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:01:23.145892 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:01:23.145904 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:01:23.146441 (+ 537us) client_negotiation.cc:769] Sending connection context
0809 13:01:23.146764 (+ 323us) client_negotiation.cc:240] Negotiation successful
0809 13:01:23.147452 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":430,"threads_started":1}
I20260809 13:01:23.150946 10788 heartbeater.cc:347] Connected to a master server at 127.21.235.126:46491
I20260809 13:01:23.151278 10788 heartbeater.cc:464] Registering TS with master...
I20260809 13:01:23.152083 10788 heartbeater.cc:511] Master 127.21.235.126:46491 requested a full tablet report, sending...
I20260809 13:01:23.154246 10567 ts_manager.cc:194] Registered new tserver with Master: cc30572649ef418e9ed3e56ac4a6095f (127.21.235.65:38825)
I20260809 13:01:23.155762 10567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:40021
WARNING: no policy specified for kudu/127.21.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
I20260809 13:01:23.198659 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:46491
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:42787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:23.544875 10795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:23.545159 10795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:23.545323 10795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:23.552587 10795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:23.552700 10795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:23.552898 10795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 13:01:23.562783 10795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:42787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:46491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:23.564168 10795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:23.565492 10795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:23.579075 10804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:23.579125 10802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:23.579188 10801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:23.579821 10795 server_base.cc:1034] running on GCE node
I20260809 13:01:23.580466 10795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:23.581462 10795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:23.582690 10795 hybrid_clock.cc:648] HybridClock initialized: now 1786280483582689 us; error 113 us; skew 500 ppm
I20260809 13:01:23.583021 10795 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:23 dist-test-slave-gp6n krb5kdc[10512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280483, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:23.586166 10795 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 13:01:23.587687 10795 webserver.cc:460] Webserver started at http://127.21.235.66:40643/ using document root <none> and password file <none>
I20260809 13:01:23.588382 10795 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:23.588531 10795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:23.588865 10795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:23.591862 10795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "ac7786c915c14281bd685f05ac38843d"
format_stamp: "Formatted at 2026-08-09 13:01:23 on dist-test-slave-gp6n"
server_key: "bc7f146383d0b059059673ccb36169e3"
server_key_iv: "15520a383dbc37896b6f40c351ef1737"
server_key_version: "encryptionkey@0"
I20260809 13:01:23.592684 10795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "ac7786c915c14281bd685f05ac38843d"
format_stamp: "Formatted at 2026-08-09 13:01:23 on dist-test-slave-gp6n"
server_key: "bc7f146383d0b059059673ccb36169e3"
server_key_iv: "15520a383dbc37896b6f40c351ef1737"
server_key_version: "encryptionkey@0"
I20260809 13:01:23.598029 10795 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:01:23.601505 10811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:23.602895 10795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:01:23.603073 10795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "ac7786c915c14281bd685f05ac38843d"
format_stamp: "Formatted at 2026-08-09 13:01:23 on dist-test-slave-gp6n"
server_key: "bc7f146383d0b059059673ccb36169e3"
server_key_iv: "15520a383dbc37896b6f40c351ef1737"
server_key_version: "encryptionkey@0"
I20260809 13:01:23.603288 10795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:23.625242 10795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:23.629637 10795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:23.630035 10795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:23.631052 10795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:01:23.632452 10795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:01:23.632563 10795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:23.632725 10795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:01:23.632800 10795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:23.666577 10795 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:37885
I20260809 13:01:23.666635 10942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:37885 every 8 connection(s)
I20260809 13:01:23.667896 10795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 13:01:23.669327 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 10795
I20260809 13:01:23.669575 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 13:01:23.670177 22445 external_mini_cluster.cc:1428] Setting key 96553e49a9fa9a732fbc59e6994b43c9
Aug 09 13:01:23 dist-test-slave-gp6n krb5kdc[10512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280483, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:01:23.688987 10945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:23.670937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:39735 (local address 127.21.235.126:46491)
0809 13:01:23.671428 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:23.671434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:23.672632 (+ 1198us) server_negotiation.cc:407] Connection header received
0809 13:01:23.674550 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:23.674556 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:23.674641 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:23.674760 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:23.676259 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:23.677028 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:23.678636 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:23.679089 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:23.679218 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:23.682628 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:23.682633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:23.682636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:23.682667 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:23.685798 (+ 3131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:23.686583 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:23.686595 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:23.686598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:23.686724 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:23.687178 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:23.687182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:23.687184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:23.687460 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:23.687635 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:23.688360 (+ 725us) server_negotiation.cc:299] Negotiation successful
0809 13:01:23.688597 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":245,"threads_started":1}
I20260809 13:01:23.690012 10946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:23.671519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:46491 (local address 127.21.235.66:39735)
0809 13:01:23.672422 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:23.672526 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:23.674241 (+ 1715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:23.675000 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:23.675008 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:23.675545 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:01:23.676094 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:23.676111 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:23.677186 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:23.677192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:23.678491 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:23.678504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:23.679324 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:23.679331 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:23.679573 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:23.680254 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:01:23.680282 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:01:23.682353 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:01:23.685942 (+ 3589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:01:23.685951 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:01:23.685968 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:01:23.686395 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:01:23.686840 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:01:23.686845 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:01:23.686849 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:01:23.687049 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:01:23.687592 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:01:23.687600 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:01:23.687978 (+ 378us) client_negotiation.cc:769] Sending connection context
0809 13:01:23.688252 (+ 274us) client_negotiation.cc:240] Negotiation successful
0809 13:01:23.688805 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":295,"threads_started":1}
I20260809 13:01:23.691692 10943 heartbeater.cc:347] Connected to a master server at 127.21.235.126:46491
I20260809 13:01:23.692018 10943 heartbeater.cc:464] Registering TS with master...
I20260809 13:01:23.692739 10943 heartbeater.cc:511] Master 127.21.235.126:46491 requested a full tablet report, sending...
I20260809 13:01:23.694198 10567 ts_manager.cc:194] Registered new tserver with Master: ac7786c915c14281bd685f05ac38843d (127.21.235.66:37885)
I20260809 13:01:23.695044 10567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:39735
WARNING: no policy specified for kudu/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
I20260809 13:01:23.739912 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:46491
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:42787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:24.079285 10949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:24.079707 10949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:24.079813 10949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:24.086866 10949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:24.086975 10949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:24.087088 10949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 13:01:24.096740 10949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:42787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:46491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:24.098074 10949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:24.099411 10949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:24.112569 10954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:24.112852 10957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:24.112979 10955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:24.113899 10949 server_base.cc:1034] running on GCE node
I20260809 13:01:24.114548 10949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:24.115450 10949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:24.116667 10949 hybrid_clock.cc:648] HybridClock initialized: now 1786280484116620 us; error 59 us; skew 500 ppm
I20260809 13:01:24.116976 10949 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:24 dist-test-slave-gp6n krb5kdc[10512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280484, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:24.120241 10949 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 13:01:24.121834 10949 webserver.cc:460] Webserver started at http://127.21.235.67:38217/ using document root <none> and password file <none>
I20260809 13:01:24.122727 10949 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:24.122876 10949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:24.123253 10949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:24.127143 10949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "2c4c64b3d1df426da19a8146d5026ec6"
format_stamp: "Formatted at 2026-08-09 13:01:24 on dist-test-slave-gp6n"
server_key: "a013f19696decc9db58842ec5d269be2"
server_key_iv: "fe73b504dcda581715593c907a7e669b"
server_key_version: "encryptionkey@0"
I20260809 13:01:24.128141 10949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "2c4c64b3d1df426da19a8146d5026ec6"
format_stamp: "Formatted at 2026-08-09 13:01:24 on dist-test-slave-gp6n"
server_key: "a013f19696decc9db58842ec5d269be2"
server_key_iv: "fe73b504dcda581715593c907a7e669b"
server_key_version: "encryptionkey@0"
I20260809 13:01:24.133505 10949 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 13:01:24.137234 10965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:24.138746 10949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:01:24.138933 10949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "2c4c64b3d1df426da19a8146d5026ec6"
format_stamp: "Formatted at 2026-08-09 13:01:24 on dist-test-slave-gp6n"
server_key: "a013f19696decc9db58842ec5d269be2"
server_key_iv: "fe73b504dcda581715593c907a7e669b"
server_key_version: "encryptionkey@0"
I20260809 13:01:24.139140 10949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:24.154000 10949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:24.158808 10949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:24.158936 10788 heartbeater.cc:503] Master 127.21.235.126:46491 was elected leader, sending a full tablet report...
I20260809 13:01:24.159168 10949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:24.160264 10949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:01:24.161823 10949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:01:24.161935 10949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:24.162101 10949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:01:24.162190 10949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:24.198371 10949 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:44021
I20260809 13:01:24.198467 11087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:44021 every 8 connection(s)
I20260809 13:01:24.199853 10949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 13:01:24.210044 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 10949
I20260809 13:01:24.210283 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 13:01:24.210875 22445 external_mini_cluster.cc:1428] Setting key 8a39dbbcbcf4e6b79fa268c6770cb1c8
Aug 09 13:01:24 dist-test-slave-gp6n krb5kdc[10512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280484, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:01:24.224526 11090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:24.202658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:45523 (local address 127.21.235.126:46491)
0809 13:01:24.203210 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:24.203219 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:24.204275 (+ 1056us) server_negotiation.cc:407] Connection header received
0809 13:01:24.206328 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:24.206333 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:24.206454 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:24.206572 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:24.208291 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.209125 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.211125 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.211782 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.211943 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:24.215905 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:24.215910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:24.215914 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:24.215949 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:24.220522 (+ 4573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:24.221467 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:24.221473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:24.221477 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:24.221665 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:24.222201 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:24.222206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:24.222210 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:24.222694 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:24.222896 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:24.223759 (+ 863us) server_negotiation.cc:299] Negotiation successful
0809 13:01:24.224044 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":248,"threads_started":1}
I20260809 13:01:24.225819 11091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:24.203273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:46491 (local address 127.21.235.67:45523)
0809 13:01:24.204094 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:24.204135 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:24.206004 (+ 1869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:24.206870 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:24.206879 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:24.207441 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:01:24.208101 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:24.208122 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.209282 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.209287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:24.210898 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:24.210911 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.212080 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.212093 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:24.212401 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:24.213150 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:01:24.213187 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:01:24.215697 (+ 2510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:01:24.220697 (+ 5000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:01:24.220709 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:01:24.220728 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:01:24.221312 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:01:24.221787 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:01:24.221803 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:01:24.221806 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:01:24.222063 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:01:24.222855 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:01:24.222864 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:01:24.223339 (+ 475us) client_negotiation.cc:769] Sending connection context
0809 13:01:24.223638 (+ 299us) client_negotiation.cc:240] Negotiation successful
0809 13:01:24.224215 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":314,"threads_started":1}
I20260809 13:01:24.227309 11088 heartbeater.cc:347] Connected to a master server at 127.21.235.126:46491
I20260809 13:01:24.227658 11088 heartbeater.cc:464] Registering TS with master...
I20260809 13:01:24.228389 11088 heartbeater.cc:511] Master 127.21.235.126:46491 requested a full tablet report, sending...
I20260809 13:01:24.229920 10567 ts_manager.cc:194] Registered new tserver with Master: 2c4c64b3d1df426da19a8146d5026ec6 (127.21.235.67:44021)
I20260809 13:01:24.230800 10567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:45523
I20260809 13:01:24.239091 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:01:24.257388 11090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:24.243419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53560 (local address 127.21.235.126:46491)
0809 13:01:24.243588 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:24.243593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:24.244128 (+ 535us) server_negotiation.cc:407] Connection header received
0809 13:01:24.244349 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:24.244355 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:24.244429 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:24.244535 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:24.245946 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.246771 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.248093 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.248493 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.248632 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:24.251611 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:24.251617 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:24.251622 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:24.251660 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:24.254383 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:24.255225 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:24.255230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:24.255234 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:24.255375 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:24.255855 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:24.255860 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:24.255871 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:24.256180 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:24.256454 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:24.256839 (+ 385us) server_negotiation.cc:299] Negotiation successful
0809 13:01:24.257050 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 13:01:24.261337 10567 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:01:24.268349 10567 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:01:24.311672 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:24.281215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59188 (local address 127.21.235.65:38825)
0809 13:01:24.301789 (+ 20574us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:24.301802 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:24.301840 (+ 38us) server_negotiation.cc:407] Connection header received
0809 13:01:24.301961 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:24.301972 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:24.302289 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:24.303622 (+ 1333us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:24.303935 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.305312 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.308184 (+ 2872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.309760 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.310069 (+ 309us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:24.310363 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:24.311126 (+ 763us) server_negotiation.cc:299] Negotiation successful
0809 13:01:24.311321 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20331,"thread_start_us":20138,"threads_started":1}
I20260809 13:01:24.311672 11103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:24.280961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:38825 (local address 127.0.0.1:59188)
0809 13:01:24.295298 (+ 14337us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:24.295358 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:24.295595 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:24.302612 (+ 7017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:24.302624 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:24.302672 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:24.303386 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:24.303404 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.305531 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.305536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:24.307976 (+ 2440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:24.308001 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.310552 (+ 2551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.310559 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:24.310861 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:24.310924 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 13:01:24.311064 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 13:01:24.311262 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13988,"thread_start_us":14896,"threads_started":1}
I20260809 13:01:24.313136 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:24.281084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:44021 (local address 127.0.0.1:40602)
0809 13:01:24.303264 (+ 22180us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:24.303290 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:24.303460 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:24.304710 (+ 1250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:24.304715 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:24.304741 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:24.305268 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:24.305282 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.307434 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.307439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:24.309863 (+ 2424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:24.309877 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.312280 (+ 2403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.312287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:24.312467 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:24.312499 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 13:01:24.312597 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 13:01:24.312719 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22019,"mutex_wait_us":1,"thread_start_us":22223,"threads_started":1}
I20260809 13:01:24.313220 11108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:24.281885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40602 (local address 127.21.235.67:44021)
0809 13:01:24.302727 (+ 20842us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:24.302738 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:24.303915 (+ 1177us) server_negotiation.cc:407] Connection header received
0809 13:01:24.304035 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:24.304043 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:24.304280 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:24.304480 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:24.305516 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.307237 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.310211 (+ 2974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.311790 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.311917 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:24.312115 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:24.312667 (+ 552us) server_negotiation.cc:299] Negotiation successful
0809 13:01:24.312856 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20624,"thread_start_us":20420,"threads_started":1}
I20260809 13:01:24.315056 11104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:24.281668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:37885 (local address 127.0.0.1:49568)
0809 13:01:24.303595 (+ 21927us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:24.303620 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:24.303747 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:24.304644 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:24.304649 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:24.304677 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:24.305236 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:24.305254 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.307167 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.307172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:24.310958 (+ 3786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:24.310973 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.314118 (+ 3145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.314127 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:24.314310 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:24.314353 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 13:01:24.314528 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 13:01:24.314653 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21802,"mutex_wait_us":77,"spinlock_wait_cycles":1920,"thread_start_us":21277,"threads_started":1}
I20260809 13:01:24.315083 11109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:24.282307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49568 (local address 127.21.235.66:37885)
0809 13:01:24.302087 (+ 19780us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:24.302100 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:24.303733 (+ 1633us) server_negotiation.cc:407] Connection header received
0809 13:01:24.303936 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:24.303944 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:24.304198 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:24.304408 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:24.305452 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.306921 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.311166 (+ 4245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.313619 (+ 2453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.313766 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:24.313974 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:24.314594 (+ 620us) server_negotiation.cc:299] Negotiation successful
0809 13:01:24.314775 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19594,"thread_start_us":19455,"threads_started":1}
I20260809 13:01:24.316493 10716 tablet_service.cc:1467] Processing CreateTablet for tablet 64e9a1fee1ed4cacaafe731b39c9cd9c (DEFAULT_TABLE table=test-table [id=6939c881b7d5495bbd1b85c5ae85ab36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:01:24.316960 11020 tablet_service.cc:1467] Processing CreateTablet for tablet 64e9a1fee1ed4cacaafe731b39c9cd9c (DEFAULT_TABLE table=test-table [id=6939c881b7d5495bbd1b85c5ae85ab36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:01:24.317967 10716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64e9a1fee1ed4cacaafe731b39c9cd9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:24.318467 11020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64e9a1fee1ed4cacaafe731b39c9cd9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:24.318897 10868 tablet_service.cc:1467] Processing CreateTablet for tablet 64e9a1fee1ed4cacaafe731b39c9cd9c (DEFAULT_TABLE table=test-table [id=6939c881b7d5495bbd1b85c5ae85ab36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:01:24.320236 10868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64e9a1fee1ed4cacaafe731b39c9cd9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:24.329780 11110 tablet_bootstrap.cc:492] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f: Bootstrap starting.
I20260809 13:01:24.333048 11110 tablet_bootstrap.cc:654] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:24.334088 11111 tablet_bootstrap.cc:492] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6: Bootstrap starting.
I20260809 13:01:24.334451 11110 log.cc:824] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:24.336711 11112 tablet_bootstrap.cc:492] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d: Bootstrap starting.
I20260809 13:01:24.338847 11111 tablet_bootstrap.cc:654] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:24.339615 11112 tablet_bootstrap.cc:654] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:24.340087 11111 log.cc:824] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:24.340759 11112 log.cc:824] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:24.341408 11110 tablet_bootstrap.cc:492] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f: No bootstrap required, opened a new log
I20260809 13:01:24.341717 11110 ts_tablet_manager.cc:1397] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260809 13:01:24.343832 11111 tablet_bootstrap.cc:492] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6: No bootstrap required, opened a new log
I20260809 13:01:24.343842 11112 tablet_bootstrap.cc:492] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d: No bootstrap required, opened a new log
I20260809 13:01:24.344097 11112 ts_tablet_manager.cc:1397] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 13:01:24.344110 11111 ts_tablet_manager.cc:1397] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.001s
I20260809 13:01:24.349632 11110 raft_consensus.cc:348] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc30572649ef418e9ed3e56ac4a6095f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38825 } } peers { permanent_uuid: "2c4c64b3d1df426da19a8146d5026ec6" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44021 } } peers { permanent_uuid: "ac7786c915c14281bd685f05ac38843d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37885 } }
I20260809 13:01:24.350006 11110 raft_consensus.cc:374] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:24.350132 11110 raft_consensus.cc:729] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc30572649ef418e9ed3e56ac4a6095f, State: Initialized, Role: FOLLOWER
I20260809 13:01:24.350800 11110 consensus_queue.cc:260] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc30572649ef418e9ed3e56ac4a6095f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38825 } } peers { permanent_uuid: "2c4c64b3d1df426da19a8146d5026ec6" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44021 } } peers { permanent_uuid: "ac7786c915c14281bd685f05ac38843d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37885 } }
I20260809 13:01:24.354182 11110 ts_tablet_manager.cc:1428] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
I20260809 13:01:24.354830 11111 raft_consensus.cc:348] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc30572649ef418e9ed3e56ac4a6095f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38825 } } peers { permanent_uuid: "2c4c64b3d1df426da19a8146d5026ec6" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44021 } } peers { permanent_uuid: "ac7786c915c14281bd685f05ac38843d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37885 } }
I20260809 13:01:24.355023 11112 raft_consensus.cc:348] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc30572649ef418e9ed3e56ac4a6095f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38825 } } peers { permanent_uuid: "2c4c64b3d1df426da19a8146d5026ec6" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44021 } } peers { permanent_uuid: "ac7786c915c14281bd685f05ac38843d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37885 } }
I20260809 13:01:24.355320 11111 raft_consensus.cc:374] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:24.355424 11111 raft_consensus.cc:729] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4c64b3d1df426da19a8146d5026ec6, State: Initialized, Role: FOLLOWER
I20260809 13:01:24.355459 11112 raft_consensus.cc:374] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:24.355552 11112 raft_consensus.cc:729] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac7786c915c14281bd685f05ac38843d, State: Initialized, Role: FOLLOWER
I20260809 13:01:24.356071 11111 consensus_queue.cc:260] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc30572649ef418e9ed3e56ac4a6095f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38825 } } peers { permanent_uuid: "2c4c64b3d1df426da19a8146d5026ec6" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44021 } } peers { permanent_uuid: "ac7786c915c14281bd685f05ac38843d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37885 } }
I20260809 13:01:24.356156 11112 consensus_queue.cc:260] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc30572649ef418e9ed3e56ac4a6095f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38825 } } peers { permanent_uuid: "2c4c64b3d1df426da19a8146d5026ec6" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44021 } } peers { permanent_uuid: "ac7786c915c14281bd685f05ac38843d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37885 } }
I20260809 13:01:24.357645 10943 heartbeater.cc:503] Master 127.21.235.126:46491 was elected leader, sending a full tablet report...
I20260809 13:01:24.357928 11111 ts_tablet_manager.cc:1428] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260809 13:01:24.358250 11112 ts_tablet_manager.cc:1428] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260809 13:01:24.358624 11088 heartbeater.cc:503] Master 127.21.235.126:46491 was elected leader, sending a full tablet report...
W20260809 13:01:24.376732 10790 tablet.cc:2369] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:01:24.420174 10944 tablet.cc:2369] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:01:24.433430 11118 raft_consensus.cc:482] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:01:24.433766 11118 raft_consensus.cc:504] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc30572649ef418e9ed3e56ac4a6095f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38825 } } peers { permanent_uuid: "2c4c64b3d1df426da19a8146d5026ec6" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44021 } } peers { permanent_uuid: "ac7786c915c14281bd685f05ac38843d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37885 } }
I20260809 13:01:24.435621 11118 leader_election.cc:288] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4c64b3d1df426da19a8146d5026ec6 (127.21.235.67:44021), ac7786c915c14281bd685f05ac38843d (127.21.235.66:37885)
I20260809 13:01:24.443126 11109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:24.436176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:38663 (local address 127.21.235.66:37885)
0809 13:01:24.436431 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:24.436441 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:24.437370 (+ 929us) server_negotiation.cc:407] Connection header received
0809 13:01:24.437550 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:24.437555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:24.437637 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:24.437742 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:24.438497 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.439201 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.441211 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.442042 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.442221 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:24.442312 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:24.442730 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 13:01:24.442867 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 13:01:24.443126 11108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:24.436102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:34269 (local address 127.21.235.67:44021)
0809 13:01:24.436344 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:24.436351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:24.437091 (+ 740us) server_negotiation.cc:407] Connection header received
0809 13:01:24.437314 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:24.437323 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:24.437418 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:24.437509 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:24.438201 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.439092 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.441050 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.442013 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.442187 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:24.442303 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:24.442713 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 13:01:24.442847 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 13:01:24.443436 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:24.436419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:37885 (local address 127.21.235.65:38663)
0809 13:01:24.437246 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:24.437263 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:24.437387 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:24.437768 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:24.437771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:24.437793 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:24.438250 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:24.438265 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.439341 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.439345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:24.441019 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:24.441035 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.442352 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.442361 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:24.442556 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:24.442587 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 13:01:24.442688 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 13:01:24.442856 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":70,"thread_start_us":489,"threads_started":1}
I20260809 13:01:24.443485 11122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:24.436085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:44021 (local address 127.21.235.65:34269)
0809 13:01:24.436957 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:24.436981 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:24.437160 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:24.437545 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:24.437551 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:24.437586 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:24.438026 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:24.438038 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.439223 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.439231 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:24.440894 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:24.440910 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.442327 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.442337 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:24.442544 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:24.442577 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 13:01:24.442687 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 13:01:24.442856 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":32,"thread_start_us":440,"threads_started":1}
I20260809 13:01:24.444550 11042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e9a1fee1ed4cacaafe731b39c9cd9c" candidate_uuid: "cc30572649ef418e9ed3e56ac4a6095f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4c64b3d1df426da19a8146d5026ec6" is_pre_election: true
I20260809 13:01:24.445015 11042 raft_consensus.cc:2454] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc30572649ef418e9ed3e56ac4a6095f in term 0.
I20260809 13:01:24.445637 10659 leader_election.cc:302] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4c64b3d1df426da19a8146d5026ec6, cc30572649ef418e9ed3e56ac4a6095f; no voters:
I20260809 13:01:24.445765 10892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e9a1fee1ed4cacaafe731b39c9cd9c" candidate_uuid: "cc30572649ef418e9ed3e56ac4a6095f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac7786c915c14281bd685f05ac38843d" is_pre_election: true
I20260809 13:01:24.446060 11118 raft_consensus.cc:2781] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:01:24.446249 11118 raft_consensus.cc:482] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:01:24.446341 11118 raft_consensus.cc:3037] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:24.446358 10892 raft_consensus.cc:2454] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc30572649ef418e9ed3e56ac4a6095f in term 0.
I20260809 13:01:24.448892 11118 raft_consensus.cc:504] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc30572649ef418e9ed3e56ac4a6095f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38825 } } peers { permanent_uuid: "2c4c64b3d1df426da19a8146d5026ec6" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44021 } } peers { permanent_uuid: "ac7786c915c14281bd685f05ac38843d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37885 } }
I20260809 13:01:24.449643 11118 leader_election.cc:288] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [CANDIDATE]: Term 1 election: Requested vote from peers 2c4c64b3d1df426da19a8146d5026ec6 (127.21.235.67:44021), ac7786c915c14281bd685f05ac38843d (127.21.235.66:37885)
I20260809 13:01:24.450091 11042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e9a1fee1ed4cacaafe731b39c9cd9c" candidate_uuid: "cc30572649ef418e9ed3e56ac4a6095f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4c64b3d1df426da19a8146d5026ec6"
I20260809 13:01:24.450172 10892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64e9a1fee1ed4cacaafe731b39c9cd9c" candidate_uuid: "cc30572649ef418e9ed3e56ac4a6095f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac7786c915c14281bd685f05ac38843d"
I20260809 13:01:24.450323 11042 raft_consensus.cc:3037] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:24.450423 10892 raft_consensus.cc:3037] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d [term 0 FOLLOWER]: Advancing to term 1
W20260809 13:01:24.451658 11089 tablet.cc:2369] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:01:24.453315 10892 raft_consensus.cc:2454] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc30572649ef418e9ed3e56ac4a6095f in term 1.
I20260809 13:01:24.453320 11042 raft_consensus.cc:2454] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc30572649ef418e9ed3e56ac4a6095f in term 1.
I20260809 13:01:24.453768 10662 leader_election.cc:302] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac7786c915c14281bd685f05ac38843d, cc30572649ef418e9ed3e56ac4a6095f; no voters:
I20260809 13:01:24.454073 11118 raft_consensus.cc:2781] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:24.454634 11118 raft_consensus.cc:686] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [term 1 LEADER]: Becoming Leader. State: Replica: cc30572649ef418e9ed3e56ac4a6095f, State: Running, Role: LEADER
I20260809 13:01:24.455132 11118 consensus_queue.cc:237] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc30572649ef418e9ed3e56ac4a6095f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38825 } } peers { permanent_uuid: "2c4c64b3d1df426da19a8146d5026ec6" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44021 } } peers { permanent_uuid: "ac7786c915c14281bd685f05ac38843d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37885 } }
I20260809 13:01:24.460031 10567 catalog_manager.cc:5495] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f reported cstate change: term changed from 0 to 1, leader changed from <none> to cc30572649ef418e9ed3e56ac4a6095f (127.21.235.65). New cstate: current_term: 1 leader_uuid: "cc30572649ef418e9ed3e56ac4a6095f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc30572649ef418e9ed3e56ac4a6095f" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 38825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4c64b3d1df426da19a8146d5026ec6" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac7786c915c14281bd685f05ac38843d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37885 } health_report { overall_health: UNKNOWN } } }
I20260809 13:01:24.501354 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:24.495659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59192 (local address 127.21.235.65:38825)
0809 13:01:24.495875 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:24.495882 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:24.495982 (+ 100us) server_negotiation.cc:407] Connection header received
0809 13:01:24.496226 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:24.496230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:24.496326 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:24.496514 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:01:24.496918 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.497635 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.499091 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.499508 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.499652 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:24.500261 (+ 609us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:01:24.500758 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:01:24.500954 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:24.501070 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 13:01:24.501114 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 13:01:24.509822 10892 raft_consensus.cc:1261] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d [term 1 FOLLOWER]: Refusing update from remote peer cc30572649ef418e9ed3e56ac4a6095f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:01:24.509828 11042 raft_consensus.cc:1261] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6 [term 1 FOLLOWER]: Refusing update from remote peer cc30572649ef418e9ed3e56ac4a6095f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:01:24.510828 11125 consensus_queue.cc:1035] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac7786c915c14281bd685f05ac38843d" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 37885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:01:24.511179 11118 consensus_queue.cc:1035] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4c64b3d1df426da19a8146d5026ec6" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:01:24.524269 11130 mvcc.cc:205] Tried to move back new op lower bound from 7316604864542109696 to 7316604864332132352. Current Snapshot: MvccSnapshot[applied={T|T < 7316604864542109696}]
I20260809 13:01:24.524659 11131 mvcc.cc:205] Tried to move back new op lower bound from 7316604864542109696 to 7316604864332132352. Current Snapshot: MvccSnapshot[applied={T|T < 7316604864542109696}]
I20260809 13:01:24.536657 10567 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:01:24.536926 10567 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:01:24.540858 10567 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49bda392e8214f8f8106f071c1a404bc: Sending DeleteTablet for 3 replicas of tablet 64e9a1fee1ed4cacaafe731b39c9cd9c
I20260809 13:01:24.542148 10716 tablet_service.cc:1514] Processing DeleteTablet for tablet 64e9a1fee1ed4cacaafe731b39c9cd9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:01:24 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:59188
I20260809 13:01:24.542284 11020 tablet_service.cc:1514] Processing DeleteTablet for tablet 64e9a1fee1ed4cacaafe731b39c9cd9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:01:24 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:40602
I20260809 13:01:24.542315 10868 tablet_service.cc:1514] Processing DeleteTablet for tablet 64e9a1fee1ed4cacaafe731b39c9cd9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:01:24 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:49568
I20260809 13:01:24.547695 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 10642
I20260809 13:01:24.556751 11142 tablet_replica.cc:330] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d: stopping tablet replica
I20260809 13:01:24.557403 11142 raft_consensus.cc:2229] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:01:24.558095 11142 raft_consensus.cc:2258] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:01:24.560436 11141 tablet_replica.cc:330] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6: stopping tablet replica
I20260809 13:01:24.561080 11142 ts_tablet_manager.cc:1905] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:01:24.561415 11141 raft_consensus.cc:2229] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:01:24.561553 11139 tablet_replica.cc:330] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f: stopping tablet replica
I20260809 13:01:24.561976 11139 raft_consensus.cc:2229] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:01:24.562045 11141 raft_consensus.cc:2258] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:01:24.562844 11139 raft_consensus.cc:2258] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:01:24.565107 11139 ts_tablet_manager.cc:1905] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:01:24.565891 11141 ts_tablet_manager.cc:1905] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:01:24.568454 11142 ts_tablet_manager.cc:1918] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:01:24.568622 11142 log.cc:1196] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/64e9a1fee1ed4cacaafe731b39c9cd9c
I20260809 13:01:24.569240 11142 ts_tablet_manager.cc:1939] T 64e9a1fee1ed4cacaafe731b39c9cd9c P ac7786c915c14281bd685f05ac38843d: Deleting consensus metadata
I20260809 13:01:24.571259 10552 catalog_manager.cc:4841] TS ac7786c915c14281bd685f05ac38843d (127.21.235.66:37885): tablet 64e9a1fee1ed4cacaafe731b39c9cd9c (table test-table [id=6939c881b7d5495bbd1b85c5ae85ab36]) successfully deleted
I20260809 13:01:24.573109 11139 ts_tablet_manager.cc:1918] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:01:24.573288 11139 log.cc:1196] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/64e9a1fee1ed4cacaafe731b39c9cd9c
I20260809 13:01:24.573850 11139 ts_tablet_manager.cc:1939] T 64e9a1fee1ed4cacaafe731b39c9cd9c P cc30572649ef418e9ed3e56ac4a6095f: Deleting consensus metadata
I20260809 13:01:24.575778 10549 catalog_manager.cc:4841] TS cc30572649ef418e9ed3e56ac4a6095f (127.21.235.65:38825): tablet 64e9a1fee1ed4cacaafe731b39c9cd9c (table test-table [id=6939c881b7d5495bbd1b85c5ae85ab36]) successfully deleted
I20260809 13:01:24.577394 11141 ts_tablet_manager.cc:1918] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:01:24.577569 11141 log.cc:1196] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/64e9a1fee1ed4cacaafe731b39c9cd9c
Aug 09 13:01:24 dist-test-slave-gp6n krb5kdc[10512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 13:01:24.578142 11141 ts_tablet_manager.cc:1939] T 64e9a1fee1ed4cacaafe731b39c9cd9c P 2c4c64b3d1df426da19a8146d5026ec6: Deleting consensus metadata
I20260809 13:01:24.579905 10547 catalog_manager.cc:4841] TS 2c4c64b3d1df426da19a8146d5026ec6 (127.21.235.67:44021): tablet 64e9a1fee1ed4cacaafe731b39c9cd9c (table test-table [id=6939c881b7d5495bbd1b85c5ae85ab36]) successfully deleted
I20260809 13:01:24.585836 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:24.549175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59196 (local address 127.21.235.65:38825)
0809 13:01:24.549424 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:24.549434 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:24.561625 (+ 12191us) server_negotiation.cc:407] Connection header received
0809 13:01:24.561967 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:24.561974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:24.562068 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:24.562144 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:24.569041 (+ 6897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.570465 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.572340 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.573121 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.573255 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:24.579074 (+ 5819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:24.579088 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:24.579106 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:24.579156 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:24.582847 (+ 3691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:24.583747 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:24.583754 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:24.583757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:24.583919 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:24.584351 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:24.584354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:24.584356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:24.584719 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:24.584960 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:24.585322 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 13:01:24.585519 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 13:01:24.586802 10783 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:24.686018 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 10642
I20260809 13:01:24.708571 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 10795
Aug 09 13:01:24 dist-test-slave-gp6n krb5kdc[10512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.66@KRBTEST.COM
I20260809 13:01:24.725152 11109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:24.709512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49572 (local address 127.21.235.66:37885)
0809 13:01:24.709735 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:24.709745 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:24.709768 (+ 23us) server_negotiation.cc:407] Connection header received
0809 13:01:24.709993 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:24.709999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:24.710078 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:24.710190 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:24.711620 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.712375 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.713711 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.714131 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.714271 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:24.718860 (+ 4589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:24.718875 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:24.718891 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:24.718926 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:24.721854 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:24.722925 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:24.722934 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:24.722936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:24.723062 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:24.723460 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:24.723463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:24.723465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:24.723851 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:24.724167 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:24.724509 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 13:01:24.724736 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 13:01:24.725978 10938 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:24.814927 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 10795
I20260809 13:01:24.834892 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 10949
Aug 09 13:01:24 dist-test-slave-gp6n krb5kdc[10512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.67@KRBTEST.COM
I20260809 13:01:24.850157 11108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:24.835760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40618 (local address 127.21.235.67:44021)
0809 13:01:24.835960 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:24.835964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:24.835983 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:01:24.836138 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:24.836141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:24.836217 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:24.836300 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:24.837594 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.838316 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.839562 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:24.839945 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:24.840038 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:24.844929 (+ 4891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:24.844945 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:24.844960 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:24.844991 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:24.847671 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:24.848305 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:24.848312 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:24.848314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:24.848444 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:24.848766 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:24.848769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:24.848771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:24.849100 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:24.849329 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:24.849616 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 13:01:24.849780 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 13:01:24.850893 11083 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:24.941798 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 10949
I20260809 13:01:24.961822 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 10527
I20260809 13:01:24.963016 10604 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:25.074071 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 10527
2026-08-09T13:01:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4571 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T13:01:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:01:25Z Disabled control of system clock
I20260809 13:01:25.169814 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:40559
--webserver_interface=127.21.235.126
--webserver_port=0
--builtin_ntp_servers=127.21.235.84:42207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:40559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 13:01:25.509918 11154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:25.510263 11154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:25.510468 11154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:25.517707 11154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:01:25.517848 11154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:25.517916 11154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:25.517998 11154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:01:25.518064 11154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:01:25.527420 11154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:42207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:40559
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:40559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:25.528705 11154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:25.530521 11154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:25.542271 11162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:25.542268 11164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:25.542574 11161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:25.543114 11154 server_base.cc:1034] running on GCE node
I20260809 13:01:25.543830 11154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:25.545192 11154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:25.546494 11154 hybrid_clock.cc:648] HybridClock initialized: now 1786280485546450 us; error 78 us; skew 500 ppm
I20260809 13:01:25.546851 11154 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:01:25.549122 11154 webserver.cc:460] Webserver started at http://127.21.235.126:46451/ using document root <none> and password file <none>
I20260809 13:01:25.549810 11154 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:25.549942 11154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:25.550259 11154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:25.553146 11154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "6ccccb40f20042199de5c962d61bc346"
format_stamp: "Formatted at 2026-08-09 13:01:25 on dist-test-slave-gp6n"
server_key: "1164f8f6706c0d53d9d474b393d2cd08"
server_key_iv: "ea439182a412ee0223778da1c390ee6f"
server_key_version: "encryptionkey@0"
I20260809 13:01:25.553951 11154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "6ccccb40f20042199de5c962d61bc346"
format_stamp: "Formatted at 2026-08-09 13:01:25 on dist-test-slave-gp6n"
server_key: "1164f8f6706c0d53d9d474b393d2cd08"
server_key_iv: "ea439182a412ee0223778da1c390ee6f"
server_key_version: "encryptionkey@0"
I20260809 13:01:25.558984 11154 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 13:01:25.562314 11171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:25.563689 11154 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:01:25.563885 11154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "6ccccb40f20042199de5c962d61bc346"
format_stamp: "Formatted at 2026-08-09 13:01:25 on dist-test-slave-gp6n"
server_key: "1164f8f6706c0d53d9d474b393d2cd08"
server_key_iv: "ea439182a412ee0223778da1c390ee6f"
server_key_version: "encryptionkey@0"
I20260809 13:01:25.564098 11154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:25.583709 11154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:25.584518 11154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:25.584810 11154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:25.603592 11154 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:40559
I20260809 13:01:25.603613 11228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:40559 every 8 connection(s)
I20260809 13:01:25.604995 11154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:01:25.606657 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 11154
I20260809 13:01:25.606837 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:01:25.607405 22445 external_mini_cluster.cc:1428] Setting key 3b4ed2dc5a462779f3fe5e99b9f8e722
I20260809 13:01:25.609737 11229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:25.621852 11232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:25.609504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56566 (local address 127.21.235.126:40559)
0809 13:01:25.610617 (+ 1113us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:25.610628 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:25.610670 (+ 42us) server_negotiation.cc:407] Connection header received
0809 13:01:25.612250 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:25.612272 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:25.612735 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:25.613081 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:25.613696 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:25.615178 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:25.617124 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:25.617803 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:25.617945 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:25.619340 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:25.619350 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:25.619365 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:25.619983 (+ 618us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:25.620070 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:25.620093 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:25.620191 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:25.620468 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 13:01:25.620844 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":366,"threads_started":1}
I20260809 13:01:25.622911 11229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346: Bootstrap starting.
I20260809 13:01:25.626180 11229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:25.627205 11229 log.cc:824] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:25.630064 11229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346: No bootstrap required, opened a new log
I20260809 13:01:25.637538 11229 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ccccb40f20042199de5c962d61bc346" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40559 } }
I20260809 13:01:25.637903 11229 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:25.638032 11229 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ccccb40f20042199de5c962d61bc346, State: Initialized, Role: FOLLOWER
I20260809 13:01:25.638674 11229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ccccb40f20042199de5c962d61bc346" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40559 } }
I20260809 13:01:25.638917 11229 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:01:25.639055 11229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:01:25.639225 11229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:25.641652 11229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ccccb40f20042199de5c962d61bc346" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40559 } }
I20260809 13:01:25.642159 11229 leader_election.cc:302] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ccccb40f20042199de5c962d61bc346; no voters:
I20260809 13:01:25.642824 11229 leader_election.cc:288] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:01:25.642992 11234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:25.643745 11234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346 [term 1 LEADER]: Becoming Leader. State: Replica: 6ccccb40f20042199de5c962d61bc346, State: Running, Role: LEADER
I20260809 13:01:25.644306 11234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ccccb40f20042199de5c962d61bc346" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40559 } }
I20260809 13:01:25.644979 11229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:01:25.647343 11236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ccccb40f20042199de5c962d61bc346. Latest consensus state: current_term: 1 leader_uuid: "6ccccb40f20042199de5c962d61bc346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ccccb40f20042199de5c962d61bc346" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40559 } } }
I20260809 13:01:25.647830 11236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:25.648500 11235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ccccb40f20042199de5c962d61bc346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ccccb40f20042199de5c962d61bc346" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 40559 } } }
I20260809 13:01:25.649597 11235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:25.651024 11244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:01:25.656342 11244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:01:25.665606 11244 catalog_manager.cc:1347] Generated new cluster ID: e54487a98c5142e6a7e1c71ac5927b1f
I20260809 13:01:25.665721 11244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:01:25.688264 11244 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:01:25.689553 11244 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:01:25.700843 11244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346: Generated new TSK 0
I20260809 13:01:25.701606 11244 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:01:25.709395 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40559
--builtin_ntp_servers=127.21.235.84:42207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 13:01:26.056929 11257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:26.057210 11257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:26.057298 11257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:26.064740 11257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:26.064845 11257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:26.064963 11257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 13:01:26.074174 11257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:42207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:26.075459 11257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:26.077136 11257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:26.090337 11263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:26.090420 11265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:26.090245 11267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:26.091132 11257 server_base.cc:1034] running on GCE node
I20260809 13:01:26.091699 11257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:26.092638 11257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:26.093873 11257 hybrid_clock.cc:648] HybridClock initialized: now 1786280486093848 us; error 58 us; skew 500 ppm
I20260809 13:01:26.094178 11257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:01:26.096506 11257 webserver.cc:460] Webserver started at http://127.21.235.65:36285/ using document root <none> and password file <none>
I20260809 13:01:26.097175 11257 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:26.097308 11257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:26.097626 11257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:26.100498 11257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "0606177e1d5e4dedbf6f159c5c66e5df"
format_stamp: "Formatted at 2026-08-09 13:01:26 on dist-test-slave-gp6n"
server_key: "c589a2ecd37e9dab193a5384a130f510"
server_key_iv: "f556178328f8032800fcfd54925deea3"
server_key_version: "encryptionkey@0"
I20260809 13:01:26.101301 11257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "0606177e1d5e4dedbf6f159c5c66e5df"
format_stamp: "Formatted at 2026-08-09 13:01:26 on dist-test-slave-gp6n"
server_key: "c589a2ecd37e9dab193a5384a130f510"
server_key_iv: "f556178328f8032800fcfd54925deea3"
server_key_version: "encryptionkey@0"
I20260809 13:01:26.106700 11257 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 13:01:26.110072 11275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:26.111512 11257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:01:26.111696 11257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "0606177e1d5e4dedbf6f159c5c66e5df"
format_stamp: "Formatted at 2026-08-09 13:01:26 on dist-test-slave-gp6n"
server_key: "c589a2ecd37e9dab193a5384a130f510"
server_key_iv: "f556178328f8032800fcfd54925deea3"
server_key_version: "encryptionkey@0"
I20260809 13:01:26.111923 11257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:26.150616 11257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:26.151705 11257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:26.152065 11257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:26.153128 11257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:01:26.154749 11257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:01:26.154871 11257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:26.155017 11257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:01:26.155107 11257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:26.187484 11257 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:43815
I20260809 13:01:26.187565 11401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:43815 every 8 connection(s)
I20260809 13:01:26.188830 11257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 13:01:26.190514 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 11257
I20260809 13:01:26.190702 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 13:01:26.191207 22445 external_mini_cluster.cc:1428] Setting key efa388c6f954b781331079ae8b1adf3a
I20260809 13:01:26.195073 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40559
--builtin_ntp_servers=127.21.235.84:42207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 13:01:26.203218 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:26.191896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:38573 (local address 127.21.235.126:40559)
0809 13:01:26.194793 (+ 2897us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:26.194801 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:26.194829 (+ 28us) server_negotiation.cc:407] Connection header received
0809 13:01:26.196082 (+ 1253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:26.196087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:26.196173 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:26.196302 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:26.197396 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:26.198137 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:26.199780 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:26.200378 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:26.200553 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:26.202210 (+ 1657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:26.202215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:26.202218 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:26.202237 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:26.202280 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:26.202293 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:26.202358 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:26.202677 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 13:01:26.202809 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2764,"thread_start_us":2645,"threads_started":1}
I20260809 13:01:26.204558 11405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:26.192548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40559 (local address 127.21.235.65:38573)
0809 13:01:26.193855 (+ 1307us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:26.193913 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:26.195809 (+ 1896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:26.196544 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:26.196553 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:26.196642 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:01:26.197230 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:26.197247 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:26.198300 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:26.198304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:26.199627 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:26.199639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:26.200642 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:26.200651 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:26.201000 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:26.201822 (+ 822us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:01:26.201827 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:01:26.201883 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:01:26.202002 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:01:26.202398 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:01:26.202440 (+ 42us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:01:26.202453 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 13:01:26.202645 (+ 192us) client_negotiation.cc:240] Negotiation successful
0809 13:01:26.203093 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":372,"threads_started":1}
I20260809 13:01:26.206193 11402 heartbeater.cc:347] Connected to a master server at 127.21.235.126:40559
I20260809 13:01:26.206574 11402 heartbeater.cc:464] Registering TS with master...
I20260809 13:01:26.207504 11402 heartbeater.cc:511] Master 127.21.235.126:40559 requested a full tablet report, sending...
I20260809 13:01:26.209836 11190 ts_manager.cc:194] Registered new tserver with Master: 0606177e1d5e4dedbf6f159c5c66e5df (127.21.235.65:43815)
I20260809 13:01:26.212091 11190 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.235.65:38573
W20260809 13:01:26.548036 11406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:26.548308 11406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:26.548398 11406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:26.555927 11406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:26.556043 11406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:26.556154 11406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 13:01:26.565317 11406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:42207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:26.566615 11406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:26.568413 11406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:26.581847 11413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:26.581986 11412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:26.581974 11415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:26.582890 11406 server_base.cc:1034] running on GCE node
I20260809 13:01:26.583511 11406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:26.584502 11406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:26.585726 11406 hybrid_clock.cc:648] HybridClock initialized: now 1786280486585654 us; error 68 us; skew 500 ppm
I20260809 13:01:26.586094 11406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:01:26.588574 11406 webserver.cc:460] Webserver started at http://127.21.235.66:40397/ using document root <none> and password file <none>
I20260809 13:01:26.589311 11406 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:26.589406 11406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:26.589716 11406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:26.592650 11406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "b6e00dbf3732435887945f0344f08e59"
format_stamp: "Formatted at 2026-08-09 13:01:26 on dist-test-slave-gp6n"
server_key: "12e165df7bc6727bf74428b96013c8d4"
server_key_iv: "8fd0fe2f40dcd27ccc97983ae48045ee"
server_key_version: "encryptionkey@0"
I20260809 13:01:26.593490 11406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "b6e00dbf3732435887945f0344f08e59"
format_stamp: "Formatted at 2026-08-09 13:01:26 on dist-test-slave-gp6n"
server_key: "12e165df7bc6727bf74428b96013c8d4"
server_key_iv: "8fd0fe2f40dcd27ccc97983ae48045ee"
server_key_version: "encryptionkey@0"
I20260809 13:01:26.598548 11406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 13:01:26.601922 11421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:26.603471 11406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:01:26.603641 11406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "b6e00dbf3732435887945f0344f08e59"
format_stamp: "Formatted at 2026-08-09 13:01:26 on dist-test-slave-gp6n"
server_key: "12e165df7bc6727bf74428b96013c8d4"
server_key_iv: "8fd0fe2f40dcd27ccc97983ae48045ee"
server_key_version: "encryptionkey@0"
I20260809 13:01:26.603901 11406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:26.622526 11406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:26.623422 11406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:26.623718 11406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:26.624778 11406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:01:26.626536 11406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:01:26.626626 11406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:26.626700 11406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:01:26.626755 11406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:26.660487 11406 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:35727
I20260809 13:01:26.660576 11557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:35727 every 8 connection(s)
I20260809 13:01:26.661818 11406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 13:01:26.669593 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 11406
I20260809 13:01:26.669780 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 13:01:26.670313 22445 external_mini_cluster.cc:1428] Setting key 38cb4ff551ec5851dd6e02934a39e2fe
I20260809 13:01:26.673602 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40559
--builtin_ntp_servers=127.21.235.84:42207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 13:01:26.675172 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:26.664486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:34291 (local address 127.21.235.126:40559)
0809 13:01:26.664665 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:26.664670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:26.665937 (+ 1267us) server_negotiation.cc:407] Connection header received
0809 13:01:26.667859 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:26.667864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:26.667938 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:26.668033 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:26.669138 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:26.670028 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:26.672024 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:26.672565 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:26.672677 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:26.674362 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:26.674367 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:26.674443 (+ 76us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:26.674472 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:26.674509 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:26.674530 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:26.674602 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:26.674863 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 13:01:26.674953 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 13:01:26.676715 11562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:26.664923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40559 (local address 127.21.235.66:34291)
0809 13:01:26.665780 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:26.665816 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:26.667543 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:26.668233 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:26.668241 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:26.668322 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:01:26.668950 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:26.668970 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:26.670180 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:26.670186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:26.671860 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:26.671874 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:26.672793 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:26.672800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:26.673098 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:26.673985 (+ 887us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:01:26.674000 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:01:26.674054 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:01:26.674155 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:01:26.674628 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:01:26.674643 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:01:26.674654 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 13:01:26.674847 (+ 193us) client_negotiation.cc:240] Negotiation successful
0809 13:01:26.675292 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":271,"threads_started":1}
I20260809 13:01:26.678191 11560 heartbeater.cc:347] Connected to a master server at 127.21.235.126:40559
I20260809 13:01:26.678580 11560 heartbeater.cc:464] Registering TS with master...
I20260809 13:01:26.679327 11560 heartbeater.cc:511] Master 127.21.235.126:40559 requested a full tablet report, sending...
I20260809 13:01:26.680800 11190 ts_manager.cc:194] Registered new tserver with Master: b6e00dbf3732435887945f0344f08e59 (127.21.235.66:35727)
I20260809 13:01:26.681614 11190 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.235.66:34291
W20260809 13:01:27.015040 11563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:27.015323 11563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:27.015470 11563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:27.022969 11563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:27.023100 11563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:27.023280 11563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 13:01:27.032189 11563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:42207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:40559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:27.033511 11563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:27.035332 11563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:27.049232 11572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:27.049322 11570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:27.049341 11569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:27.050182 11563 server_base.cc:1034] running on GCE node
I20260809 13:01:27.050922 11563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:27.052007 11563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:27.053258 11563 hybrid_clock.cc:648] HybridClock initialized: now 1786280487053211 us; error 66 us; skew 500 ppm
I20260809 13:01:27.053623 11563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:01:27.056443 11563 webserver.cc:460] Webserver started at http://127.21.235.67:38895/ using document root <none> and password file <none>
I20260809 13:01:27.057163 11563 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:27.057296 11563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:27.057658 11563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:27.060683 11563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "35100224477b420393732afd17e6d8ac"
format_stamp: "Formatted at 2026-08-09 13:01:27 on dist-test-slave-gp6n"
server_key: "36b49c8f4cee8131100bc1b27a050cee"
server_key_iv: "0b29e6353adeb5b440100b34bce098f1"
server_key_version: "encryptionkey@0"
I20260809 13:01:27.061522 11563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "35100224477b420393732afd17e6d8ac"
format_stamp: "Formatted at 2026-08-09 13:01:27 on dist-test-slave-gp6n"
server_key: "36b49c8f4cee8131100bc1b27a050cee"
server_key_iv: "0b29e6353adeb5b440100b34bce098f1"
server_key_version: "encryptionkey@0"
I20260809 13:01:27.066861 11563 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 13:01:27.070585 11578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:27.071945 11563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:01:27.072134 11563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "35100224477b420393732afd17e6d8ac"
format_stamp: "Formatted at 2026-08-09 13:01:27 on dist-test-slave-gp6n"
server_key: "36b49c8f4cee8131100bc1b27a050cee"
server_key_iv: "0b29e6353adeb5b440100b34bce098f1"
server_key_version: "encryptionkey@0"
I20260809 13:01:27.072362 11563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:27.096865 11563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:27.097795 11563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:27.098138 11563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:27.099205 11563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:01:27.100728 11563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:01:27.100853 11563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:27.100988 11563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:01:27.101079 11563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:27.134696 11563 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:39511
I20260809 13:01:27.134788 11703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:39511 every 8 connection(s)
I20260809 13:01:27.136070 11563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 13:01:27.145736 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 11563
I20260809 13:01:27.145913 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 13:01:27.146562 22445 external_mini_cluster.cc:1428] Setting key 1c9eb6a566c4ab1b3a21eb98502f26c4
I20260809 13:01:27.149300 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.138732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:54313 (local address 127.21.235.126:40559)
0809 13:01:27.138924 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:27.138928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:27.140472 (+ 1544us) server_negotiation.cc:407] Connection header received
0809 13:01:27.142299 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:27.142305 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:27.142442 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:27.142566 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:27.143651 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.144416 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.146192 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.146835 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.146985 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.148380 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:27.148386 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:27.148391 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:27.148417 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:27.148449 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:27.148462 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:27.148566 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:27.148783 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 13:01:27.148892 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 13:01:27.150267 11706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.139298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:40559 (local address 127.21.235.67:54313)
0809 13:01:27.140274 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:27.140312 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:27.142017 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:27.142802 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:27.142811 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:27.142905 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:01:27.143480 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:27.143497 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.144586 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.144590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:27.146054 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:27.146066 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.147075 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.147083 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:27.147315 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.148055 (+ 740us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:01:27.148058 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:01:27.148096 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:01:27.148177 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:01:27.148564 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:01:27.148576 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:01:27.148586 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 13:01:27.148773 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 13:01:27.149172 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":303,"threads_started":1}
I20260809 13:01:27.151675 11704 heartbeater.cc:347] Connected to a master server at 127.21.235.126:40559
I20260809 13:01:27.152021 11704 heartbeater.cc:464] Registering TS with master...
I20260809 13:01:27.152801 11704 heartbeater.cc:511] Master 127.21.235.126:40559 requested a full tablet report, sending...
I20260809 13:01:27.154193 11190 ts_manager.cc:194] Registered new tserver with Master: 35100224477b420393732afd17e6d8ac (127.21.235.67:39511)
I20260809 13:01:27.155061 11190 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.235.67:54313
I20260809 13:01:27.161617 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:01:27.172525 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.166237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56572 (local address 127.21.235.126:40559)
0809 13:01:27.166525 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:27.166534 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:27.166901 (+ 367us) server_negotiation.cc:407] Connection header received
0809 13:01:27.167092 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:27.167096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:27.167158 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:27.167241 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:27.167869 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.168709 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.170122 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.170595 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.170691 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.171778 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:27.171783 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:27.171786 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:27.171803 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:27.171845 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:27.171856 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:27.171973 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:27.172129 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 13:01:27.172205 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260809 13:01:27.176652 11190 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:01:27.183495 11190 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:01:27.215466 11402 heartbeater.cc:503] Master 127.21.235.126:40559 was elected leader, sending a full tablet report...
I20260809 13:01:27.220504 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.196309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:39511 (local address 127.0.0.1:37264)
0809 13:01:27.211412 (+ 15103us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:27.211443 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:27.211611 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:27.212969 (+ 1358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:27.212977 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:27.213008 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:27.213470 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:27.213486 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.215051 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.215058 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:27.217558 (+ 2500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:27.217571 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.219591 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.219597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:27.219748 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.219781 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 13:01:27.219923 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 13:01:27.220138 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14907,"thread_start_us":1072,"threads_started":1}
I20260809 13:01:27.220526 11720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.196929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37264 (local address 127.21.235.67:39511)
0809 13:01:27.210736 (+ 13807us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:27.210743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:27.211584 (+ 841us) server_negotiation.cc:407] Connection header received
0809 13:01:27.212039 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:27.212053 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:27.212395 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:27.212589 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:27.213730 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.214923 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.217739 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.218975 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.219108 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.219449 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:27.219980 (+ 531us) server_negotiation.cc:299] Negotiation successful
0809 13:01:27.220155 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13627,"thread_start_us":13472,"threads_started":1}
I20260809 13:01:27.220942 11718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.196127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:35727 (local address 127.0.0.1:45912)
0809 13:01:27.203704 (+ 7577us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:27.203766 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:27.203983 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:27.213606 (+ 9623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:27.213610 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:27.213634 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:27.214055 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:27.214069 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.216433 (+ 2364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.216441 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:27.218207 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:27.218220 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.220113 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.220120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:27.220320 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.220354 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 13:01:27.220432 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 13:01:27.220569 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1683,"mutex_wait_us":72,"thread_start_us":15581,"threads_started":1}
I20260809 13:01:27.222136 11717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.196306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45912 (local address 127.21.235.66:35727)
0809 13:01:27.213022 (+ 16716us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:27.213030 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:27.213056 (+ 26us) server_negotiation.cc:407] Connection header received
0809 13:01:27.213148 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:27.213155 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:27.213365 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:27.214286 (+ 921us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:27.214573 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.216223 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.218444 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.219873 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.221307 (+ 1434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.221558 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:27.221634 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 13:01:27.221807 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16538,"thread_start_us":17173,"threads_started":1}
I20260809 13:01:27.223424 11721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.197578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56810 (local address 127.21.235.65:43815)
0809 13:01:27.212201 (+ 14623us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:27.212214 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:27.212955 (+ 741us) server_negotiation.cc:407] Connection header received
0809 13:01:27.213076 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:27.213084 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:27.213378 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:27.213595 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:27.214555 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.216370 (+ 1815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.220356 (+ 3986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.221946 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.222082 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.222350 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:27.222888 (+ 538us) server_negotiation.cc:299] Negotiation successful
0809 13:01:27.223074 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14381,"thread_start_us":14239,"threads_started":1}
I20260809 13:01:27.223922 11722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.197768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:43815 (local address 127.0.0.1:56810)
0809 13:01:27.212654 (+ 14886us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:27.212682 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:27.212814 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:27.213844 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:27.213850 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:27.213895 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:27.214321 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:27.214333 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.216522 (+ 2189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.216529 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:27.220139 (+ 3610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:27.220151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.222501 (+ 2350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.222507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:27.222676 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.222707 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 13:01:27.222793 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 13:01:27.222933 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14761,"mutex_wait_us":137,"thread_start_us":14566,"threads_started":1}
I20260809 13:01:27.224994 11632 tablet_service.cc:1467] Processing CreateTablet for tablet d4c0f0f65ea04dd7beefa48d0aa01b33 (DEFAULT_TABLE table=test-table [id=11d04faf21244716a14337c41386c08f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:01:27.225785 11478 tablet_service.cc:1467] Processing CreateTablet for tablet d4c0f0f65ea04dd7beefa48d0aa01b33 (DEFAULT_TABLE table=test-table [id=11d04faf21244716a14337c41386c08f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:01:27.226816 11632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4c0f0f65ea04dd7beefa48d0aa01b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:27.227240 11478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4c0f0f65ea04dd7beefa48d0aa01b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:27.227640 11329 tablet_service.cc:1467] Processing CreateTablet for tablet d4c0f0f65ea04dd7beefa48d0aa01b33 (DEFAULT_TABLE table=test-table [id=11d04faf21244716a14337c41386c08f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:01:27.229000 11329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4c0f0f65ea04dd7beefa48d0aa01b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:27.238616 11724 tablet_bootstrap.cc:492] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59: Bootstrap starting.
I20260809 13:01:27.241618 11724 tablet_bootstrap.cc:654] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:27.242728 11725 tablet_bootstrap.cc:492] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac: Bootstrap starting.
I20260809 13:01:27.243075 11724 log.cc:824] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:27.245519 11726 tablet_bootstrap.cc:492] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df: Bootstrap starting.
I20260809 13:01:27.247553 11725 tablet_bootstrap.cc:654] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:27.248621 11726 tablet_bootstrap.cc:654] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:27.249115 11725 log.cc:824] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:27.249624 11726 log.cc:824] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:27.252497 11726 tablet_bootstrap.cc:492] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df: No bootstrap required, opened a new log
I20260809 13:01:27.252702 11725 tablet_bootstrap.cc:492] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac: No bootstrap required, opened a new log
I20260809 13:01:27.252734 11724 tablet_bootstrap.cc:492] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59: No bootstrap required, opened a new log
I20260809 13:01:27.252799 11726 ts_tablet_manager.cc:1397] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 13:01:27.252977 11725 ts_tablet_manager.cc:1397] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260809 13:01:27.252977 11724 ts_tablet_manager.cc:1397] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
I20260809 13:01:27.260860 11726 raft_consensus.cc:348] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e00dbf3732435887945f0344f08e59" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 35727 } } peers { permanent_uuid: "35100224477b420393732afd17e6d8ac" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39511 } } peers { permanent_uuid: "0606177e1d5e4dedbf6f159c5c66e5df" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 43815 } }
I20260809 13:01:27.261253 11726 raft_consensus.cc:374] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:27.261377 11726 raft_consensus.cc:729] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0606177e1d5e4dedbf6f159c5c66e5df, State: Initialized, Role: FOLLOWER
I20260809 13:01:27.262025 11726 consensus_queue.cc:260] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e00dbf3732435887945f0344f08e59" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 35727 } } peers { permanent_uuid: "35100224477b420393732afd17e6d8ac" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39511 } } peers { permanent_uuid: "0606177e1d5e4dedbf6f159c5c66e5df" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 43815 } }
I20260809 13:01:27.263994 11726 ts_tablet_manager.cc:1428] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df: Time spent starting tablet: real 0.011s user 0.006s sys 0.005s
I20260809 13:01:27.263926 11724 raft_consensus.cc:348] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e00dbf3732435887945f0344f08e59" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 35727 } } peers { permanent_uuid: "35100224477b420393732afd17e6d8ac" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39511 } } peers { permanent_uuid: "0606177e1d5e4dedbf6f159c5c66e5df" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 43815 } }
I20260809 13:01:27.263926 11725 raft_consensus.cc:348] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e00dbf3732435887945f0344f08e59" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 35727 } } peers { permanent_uuid: "35100224477b420393732afd17e6d8ac" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39511 } } peers { permanent_uuid: "0606177e1d5e4dedbf6f159c5c66e5df" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 43815 } }
I20260809 13:01:27.264412 11725 raft_consensus.cc:374] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:27.264411 11724 raft_consensus.cc:374] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:27.264528 11725 raft_consensus.cc:729] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35100224477b420393732afd17e6d8ac, State: Initialized, Role: FOLLOWER
I20260809 13:01:27.264528 11724 raft_consensus.cc:729] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6e00dbf3732435887945f0344f08e59, State: Initialized, Role: FOLLOWER
I20260809 13:01:27.265122 11725 consensus_queue.cc:260] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e00dbf3732435887945f0344f08e59" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 35727 } } peers { permanent_uuid: "35100224477b420393732afd17e6d8ac" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39511 } } peers { permanent_uuid: "0606177e1d5e4dedbf6f159c5c66e5df" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 43815 } }
I20260809 13:01:27.265120 11724 consensus_queue.cc:260] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e00dbf3732435887945f0344f08e59" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 35727 } } peers { permanent_uuid: "35100224477b420393732afd17e6d8ac" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39511 } } peers { permanent_uuid: "0606177e1d5e4dedbf6f159c5c66e5df" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 43815 } }
I20260809 13:01:27.266642 11560 heartbeater.cc:503] Master 127.21.235.126:40559 was elected leader, sending a full tablet report...
I20260809 13:01:27.266698 11704 heartbeater.cc:503] Master 127.21.235.126:40559 was elected leader, sending a full tablet report...
I20260809 13:01:27.266994 11724 ts_tablet_manager.cc:1428] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260809 13:01:27.267319 11725 ts_tablet_manager.cc:1428] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260809 13:01:27.317829 11731 raft_consensus.cc:482] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:01:27.318282 11731 raft_consensus.cc:504] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e00dbf3732435887945f0344f08e59" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 35727 } } peers { permanent_uuid: "35100224477b420393732afd17e6d8ac" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39511 } } peers { permanent_uuid: "0606177e1d5e4dedbf6f159c5c66e5df" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 43815 } }
I20260809 13:01:27.320556 11731 leader_election.cc:288] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35100224477b420393732afd17e6d8ac (127.21.235.67:39511), 0606177e1d5e4dedbf6f159c5c66e5df (127.21.235.65:43815)
I20260809 13:01:27.330130 11720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.321282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:45823 (local address 127.21.235.67:39511)
0809 13:01:27.321478 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:27.321485 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:27.322346 (+ 861us) server_negotiation.cc:407] Connection header received
0809 13:01:27.322634 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:27.322642 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:27.322733 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:27.322834 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:27.323497 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.324540 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.327580 (+ 3040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.328883 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.329063 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.329137 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:27.329742 (+ 605us) server_negotiation.cc:299] Negotiation successful
0809 13:01:27.329880 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 13:01:27.330130 11721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.321266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:57979 (local address 127.21.235.65:43815)
0809 13:01:27.321500 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:27.321508 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:27.322969 (+ 1461us) server_negotiation.cc:407] Connection header received
0809 13:01:27.323234 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:27.323239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:27.323307 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:27.323458 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:27.323932 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.324868 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.327580 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.328906 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.329063 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.329137 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:27.329715 (+ 578us) server_negotiation.cc:299] Negotiation successful
0809 13:01:27.329852 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260809 13:01:27.330242 11733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.321081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:39511 (local address 127.21.235.66:45823)
0809 13:01:27.322222 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:27.322238 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:27.322437 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:27.322875 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:27.322881 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:27.322914 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:27.323323 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:27.323332 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.324692 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.324696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:27.327325 (+ 2629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:27.327343 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.329125 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.329136 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:27.329429 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.329559 (+ 130us) client_negotiation.cc:769] Sending connection context
0809 13:01:27.329659 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 13:01:27.329861 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1017,"mutex_wait_us":270,"thread_start_us":819,"threads_started":1}
I20260809 13:01:27.330524 11734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.321081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:43815 (local address 127.21.235.66:57979)
0809 13:01:27.322869 (+ 1788us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:27.322884 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:27.323086 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:27.323465 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:27.323469 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:27.323492 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:27.323772 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:27.323779 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.325011 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.325018 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:27.327324 (+ 2306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:27.327341 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.329146 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.329151 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:27.329433 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.329559 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 13:01:27.329658 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 13:01:27.329861 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1650,"mutex_wait_us":224,"thread_start_us":1382,"threads_started":1}
I20260809 13:01:27.330766 11730 raft_consensus.cc:482] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:01:27.331099 11730 raft_consensus.cc:504] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e00dbf3732435887945f0344f08e59" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 35727 } } peers { permanent_uuid: "35100224477b420393732afd17e6d8ac" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39511 } } peers { permanent_uuid: "0606177e1d5e4dedbf6f159c5c66e5df" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 43815 } }
I20260809 13:01:27.331830 11351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4c0f0f65ea04dd7beefa48d0aa01b33" candidate_uuid: "b6e00dbf3732435887945f0344f08e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0606177e1d5e4dedbf6f159c5c66e5df" is_pre_election: true
I20260809 13:01:27.331861 11652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4c0f0f65ea04dd7beefa48d0aa01b33" candidate_uuid: "b6e00dbf3732435887945f0344f08e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35100224477b420393732afd17e6d8ac" is_pre_election: true
I20260809 13:01:27.332298 11351 raft_consensus.cc:2454] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6e00dbf3732435887945f0344f08e59 in term 0.
I20260809 13:01:27.332448 11652 raft_consensus.cc:2454] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6e00dbf3732435887945f0344f08e59 in term 0.
I20260809 13:01:27.333019 11423 leader_election.cc:302] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0606177e1d5e4dedbf6f159c5c66e5df, b6e00dbf3732435887945f0344f08e59; no voters:
I20260809 13:01:27.333232 11730 leader_election.cc:288] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6e00dbf3732435887945f0344f08e59 (127.21.235.66:35727), 35100224477b420393732afd17e6d8ac (127.21.235.67:39511)
I20260809 13:01:27.333448 11731 raft_consensus.cc:2781] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:01:27.333607 11731 raft_consensus.cc:482] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:01:27.333701 11731 raft_consensus.cc:3037] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:27.337292 11731 raft_consensus.cc:504] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e00dbf3732435887945f0344f08e59" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 35727 } } peers { permanent_uuid: "35100224477b420393732afd17e6d8ac" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39511 } } peers { permanent_uuid: "0606177e1d5e4dedbf6f159c5c66e5df" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 43815 } }
I20260809 13:01:27.338016 11731 leader_election.cc:288] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [CANDIDATE]: Term 1 election: Requested vote from peers 35100224477b420393732afd17e6d8ac (127.21.235.67:39511), 0606177e1d5e4dedbf6f159c5c66e5df (127.21.235.65:43815)
I20260809 13:01:27.338392 11652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4c0f0f65ea04dd7beefa48d0aa01b33" candidate_uuid: "b6e00dbf3732435887945f0344f08e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35100224477b420393732afd17e6d8ac"
I20260809 13:01:27.338660 11652 raft_consensus.cc:3037] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:27.339525 11351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4c0f0f65ea04dd7beefa48d0aa01b33" candidate_uuid: "b6e00dbf3732435887945f0344f08e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0606177e1d5e4dedbf6f159c5c66e5df"
I20260809 13:01:27.339752 11351 raft_consensus.cc:3037] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:27.342003 11652 raft_consensus.cc:2454] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6e00dbf3732435887945f0344f08e59 in term 1.
I20260809 13:01:27.342469 11425 leader_election.cc:302] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35100224477b420393732afd17e6d8ac, b6e00dbf3732435887945f0344f08e59; no voters:
I20260809 13:01:27.342764 11731 raft_consensus.cc:2781] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:27.342907 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.333736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:39511 (local address 127.21.235.65:57439)
0809 13:01:27.336018 (+ 2282us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:27.336036 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:27.336167 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:27.336460 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:27.336465 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:27.336492 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:27.336806 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:27.336816 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.337805 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.337810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:27.340042 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:27.340056 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.342043 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.342058 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:27.342254 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.342298 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 13:01:27.342390 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 13:01:27.342543 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2162,"thread_start_us":1642,"threads_started":1}
I20260809 13:01:27.343225 11351 raft_consensus.cc:2454] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6e00dbf3732435887945f0344f08e59 in term 1.
I20260809 13:01:27.343549 11720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.334223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:57439 (local address 127.21.235.67:39511)
0809 13:01:27.334459 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:27.334467 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:27.336127 (+ 1660us) server_negotiation.cc:407] Connection header received
0809 13:01:27.336279 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:27.336282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:27.336338 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:27.336417 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:27.336941 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.337666 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.340281 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.341609 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.341750 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.341862 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:27.343160 (+ 1298us) server_negotiation.cc:299] Negotiation successful
0809 13:01:27.343289 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 13:01:27.343994 11652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4c0f0f65ea04dd7beefa48d0aa01b33" candidate_uuid: "0606177e1d5e4dedbf6f159c5c66e5df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35100224477b420393732afd17e6d8ac" is_pre_election: true
I20260809 13:01:27.344304 11652 raft_consensus.cc:2379] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0606177e1d5e4dedbf6f159c5c66e5df in current term 1: Already voted for candidate b6e00dbf3732435887945f0344f08e59 in this term.
I20260809 13:01:27.345423 11731 raft_consensus.cc:686] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [term 1 LEADER]: Becoming Leader. State: Replica: b6e00dbf3732435887945f0344f08e59, State: Running, Role: LEADER
I20260809 13:01:27.346374 11737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.333577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:35727 (local address 127.21.235.65:46085)
0809 13:01:27.334968 (+ 1391us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:27.334993 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:27.335149 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:27.339207 (+ 4058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:27.339212 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:27.339237 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:27.339710 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:27.339722 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.341616 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.341621 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:27.343817 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:27.343833 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.345713 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.345717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:27.345830 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.345850 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 13:01:27.345962 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 13:01:27.346086 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1218,"mutex_wait_us":1,"thread_start_us":2134,"threads_started":1}
I20260809 13:01:27.346472 11717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.335355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:46085 (local address 127.21.235.66:35727)
0809 13:01:27.338351 (+ 2996us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:27.338357 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:27.338375 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:01:27.338469 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:27.338475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:27.338569 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:27.338660 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:27.339898 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.340974 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.344065 (+ 3091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.345341 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.345483 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.345555 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:27.346031 (+ 476us) server_negotiation.cc:299] Negotiation successful
0809 13:01:27.346162 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2855}
I20260809 13:01:27.346289 11731 consensus_queue.cc:237] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e00dbf3732435887945f0344f08e59" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 35727 } } peers { permanent_uuid: "35100224477b420393732afd17e6d8ac" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39511 } } peers { permanent_uuid: "0606177e1d5e4dedbf6f159c5c66e5df" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 43815 } }
I20260809 13:01:27.347301 11500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4c0f0f65ea04dd7beefa48d0aa01b33" candidate_uuid: "0606177e1d5e4dedbf6f159c5c66e5df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6e00dbf3732435887945f0344f08e59" is_pre_election: true
I20260809 13:01:27.348346 11280 leader_election.cc:302] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0606177e1d5e4dedbf6f159c5c66e5df; no voters: 35100224477b420393732afd17e6d8ac, b6e00dbf3732435887945f0344f08e59
I20260809 13:01:27.348738 11730 raft_consensus.cc:2726] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 13:01:27.352079 11190 catalog_manager.cc:5495] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6e00dbf3732435887945f0344f08e59 (127.21.235.66). New cstate: current_term: 1 leader_uuid: "b6e00dbf3732435887945f0344f08e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e00dbf3732435887945f0344f08e59" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 35727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35100224477b420393732afd17e6d8ac" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0606177e1d5e4dedbf6f159c5c66e5df" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 43815 } health_report { overall_health: UNKNOWN } } }
W20260809 13:01:27.387830 11705 tablet.cc:2369] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:01:27.402333 11717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.396338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45920 (local address 127.21.235.66:35727)
0809 13:01:27.396594 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:27.396601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:27.396729 (+ 128us) server_negotiation.cc:407] Connection header received
0809 13:01:27.397001 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:27.397005 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:27.397087 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:27.397251 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:01:27.397678 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.398539 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.399958 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.400397 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.400543 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.401198 (+ 655us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:01:27.401713 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:01:27.401920 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:27.402009 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 13:01:27.402052 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 13:01:27.410890 11351 raft_consensus.cc:1261] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df [term 1 FOLLOWER]: Refusing update from remote peer b6e00dbf3732435887945f0344f08e59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:01:27.411018 11652 raft_consensus.cc:1261] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac [term 1 FOLLOWER]: Refusing update from remote peer b6e00dbf3732435887945f0344f08e59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:01:27.411909 11738 consensus_queue.cc:1035] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0606177e1d5e4dedbf6f159c5c66e5df" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 43815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:01:27.412285 11731 consensus_queue.cc:1035] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35100224477b420393732afd17e6d8ac" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 39511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 13:01:27.413928 11561 tablet.cc:2369] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:01:27.424973 11741 mvcc.cc:205] Tried to move back new op lower bound from 7316604876424736768 to 7316604876177768448. Current Snapshot: MvccSnapshot[applied={T|T < 7316604876424736768}]
I20260809 13:01:27.440162 11189 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:01:27.440462 11189 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:01:27.444626 11189 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ccccb40f20042199de5c962d61bc346: Sending DeleteTablet for 3 replicas of tablet d4c0f0f65ea04dd7beefa48d0aa01b33
I20260809 13:01:27.445758 11632 tablet_service.cc:1514] Processing DeleteTablet for tablet d4c0f0f65ea04dd7beefa48d0aa01b33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:01:27 UTC) from {username='slave'} at 127.0.0.1:37264
I20260809 13:01:27.445786 11478 tablet_service.cc:1514] Processing DeleteTablet for tablet d4c0f0f65ea04dd7beefa48d0aa01b33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:01:27 UTC) from {username='slave'} at 127.0.0.1:45912
I20260809 13:01:27.446559 11329 tablet_service.cc:1514] Processing DeleteTablet for tablet d4c0f0f65ea04dd7beefa48d0aa01b33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:01:27 UTC) from {username='slave'} at 127.0.0.1:56810
I20260809 13:01:27.446807 11752 tablet_replica.cc:330] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac: stopping tablet replica
I20260809 13:01:27.447337 11752 raft_consensus.cc:2229] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:01:27.447906 11752 raft_consensus.cc:2258] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:01:27.449090 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 11257
I20260809 13:01:27.450383 11752 ts_tablet_manager.cc:1905] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:01:27.454126 11754 tablet_replica.cc:330] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df: stopping tablet replica
I20260809 13:01:27.455662 11754 raft_consensus.cc:2229] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:01:27.456347 11754 raft_consensus.cc:2258] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:01:27.458437 11752 ts_tablet_manager.cc:1918] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:01:27.458650 11752 log.cc:1196] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/d4c0f0f65ea04dd7beefa48d0aa01b33
I20260809 13:01:27.459360 11752 ts_tablet_manager.cc:1939] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 35100224477b420393732afd17e6d8ac: Deleting consensus metadata
I20260809 13:01:27.460798 11754 ts_tablet_manager.cc:1905] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:01:27.461031 11753 tablet_replica.cc:330] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59: stopping tablet replica
I20260809 13:01:27.461490 11753 raft_consensus.cc:2229] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:01:27.461826 11176 catalog_manager.cc:4841] TS 35100224477b420393732afd17e6d8ac (127.21.235.67:39511): tablet d4c0f0f65ea04dd7beefa48d0aa01b33 (table test-table [id=11d04faf21244716a14337c41386c08f]) successfully deleted
I20260809 13:01:27.462116 11753 raft_consensus.cc:2258] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:01:27.463505 11721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.450561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56824 (local address 127.21.235.65:43815)
0809 13:01:27.450749 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:27.450756 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:27.455384 (+ 4628us) server_negotiation.cc:407] Connection header received
0809 13:01:27.455533 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:27.455539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:27.455643 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:27.455747 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:27.456369 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.457583 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.459572 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.460199 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.460295 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.461824 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:27.461836 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:27.461853 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:27.462699 (+ 846us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:27.462754 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:27.462773 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:27.462837 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:27.463048 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 13:01:27.463154 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 13:01:27.464147 11753 ts_tablet_manager.cc:1905] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:01:27.464285 11397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:27.471787 11753 ts_tablet_manager.cc:1918] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:01:27.471966 11753 log.cc:1196] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/d4c0f0f65ea04dd7beefa48d0aa01b33
I20260809 13:01:27.472537 11753 ts_tablet_manager.cc:1939] T d4c0f0f65ea04dd7beefa48d0aa01b33 P b6e00dbf3732435887945f0344f08e59: Deleting consensus metadata
I20260809 13:01:27.474313 11177 catalog_manager.cc:4841] TS b6e00dbf3732435887945f0344f08e59 (127.21.235.66:35727): tablet d4c0f0f65ea04dd7beefa48d0aa01b33 (table test-table [id=11d04faf21244716a14337c41386c08f]) successfully deleted
I20260809 13:01:27.559362 11754 ts_tablet_manager.cc:1918] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:01:27.559604 11754 log.cc:1196] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/d4c0f0f65ea04dd7beefa48d0aa01b33
I20260809 13:01:27.560243 11754 ts_tablet_manager.cc:1939] T d4c0f0f65ea04dd7beefa48d0aa01b33 P 0606177e1d5e4dedbf6f159c5c66e5df: Deleting consensus metadata
I20260809 13:01:27.560318 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 11257
W20260809 13:01:27.588363 11174 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.235.65:43815 (error 108)
I20260809 13:01:27.588547 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 11406
I20260809 13:01:27.590458 11722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.589916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:43815 (local address 127.0.0.1:56830)
0809 13:01:27.590115 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:27.590254 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:43815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260809 13:01:27.590847 11174 catalog_manager.cc:4584] TS 0606177e1d5e4dedbf6f159c5c66e5df (127.21.235.65:43815): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d4c0f0f65ea04dd7beefa48d0aa01b33: Network error: Client connection negotiation failed: client connection to 127.21.235.65:43815: connect: Connection refused (error 111)
I20260809 13:01:27.596516 11717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.589999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45924 (local address 127.21.235.66:35727)
0809 13:01:27.590167 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:27.590174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:27.590194 (+ 20us) server_negotiation.cc:407] Connection header received
0809 13:01:27.590260 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:27.590265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:27.590443 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:27.590577 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:27.591204 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.591926 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.593234 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.593701 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.593854 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.595114 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:27.595122 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:27.595135 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:27.595732 (+ 597us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:27.595781 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:27.595801 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:27.595924 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:27.596065 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 13:01:27.596168 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 13:01:27.597213 11553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:27.644428 11722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.643941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:43815 (local address 127.0.0.1:56836)
0809 13:01:27.644160 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:27.644249 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:43815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 13:01:27.645561 11722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.645159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:43815 (local address 127.0.0.1:56842)
0809 13:01:27.645375 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:27.645445 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:43815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 13:01:27.706373 11722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.705762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:43815 (local address 127.0.0.1:56844)
0809 13:01:27.705972 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:27.706090 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:43815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 13:01:27.707350 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 11406
I20260809 13:01:27.709599 11722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.707159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:43815 (local address 127.0.0.1:56852)
0809 13:01:27.709413 (+ 2254us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:27.709487 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:43815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2159}
I20260809 13:01:27.728253 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 11563
I20260809 13:01:27.735531 11720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.728997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37272 (local address 127.21.235.67:39511)
0809 13:01:27.729204 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:27.729210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:27.729230 (+ 20us) server_negotiation.cc:407] Connection header received
0809 13:01:27.729477 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:27.729483 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:27.729582 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:27.729689 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:27.730115 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.730908 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.732247 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:27.732682 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:27.732832 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:27.733868 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:27.733883 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:27.733897 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:27.734752 (+ 855us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:27.734827 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:27.734855 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:27.734966 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:27.735116 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 13:01:27.735217 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 13:01:27.736320 11699 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:27.790558 11722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.789945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:43815 (local address 127.0.0.1:56868)
0809 13:01:27.790190 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:27.790311 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:43815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 13:01:27.791719 11722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:27.791309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:43815 (local address 127.0.0.1:56880)
0809 13:01:27.791509 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:27.791616 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:43815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260809 13:01:27.827009 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 11563
I20260809 13:01:27.846772 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 11154
I20260809 13:01:27.847920 11223 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:27.960307 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 11154
2026-08-09T13:01:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2886 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:01:28 dist-test-slave-gp6n krb5kdc[11763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:01:28 dist-test-slave-gp6n krb5kdc[11763](info): set up 2 sockets
Aug 09 13:01:28 dist-test-slave-gp6n krb5kdc[11763](info): commencing operation
krb5kdc: starting...
W20260809 13:01:29.165328 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:01:29 dist-test-slave-gp6n krb5kdc[11763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:01:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:01:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 13:01:29.474733 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:33125
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:35039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:33125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:29.820338 11777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:29.820683 11777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:29.820847 11777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:29.828295 11777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:01:29.828416 11777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:29.828532 11777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:29.828637 11777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:01:29.828723 11777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:01:29.838842 11777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:35039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:33125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:33125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:29.840273 11777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:29.841666 11777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:29.854118 11784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:29.854051 11785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:29.854090 11787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:29.855355 11777 server_base.cc:1034] running on GCE node
I20260809 13:01:29.856158 11777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:29.857676 11777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:29.858932 11777 hybrid_clock.cc:648] HybridClock initialized: now 1786280489858872 us; error 44 us; skew 500 ppm
I20260809 13:01:29.859309 11777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:29 dist-test-slave-gp6n krb5kdc[11763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280489, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:29.862957 11777 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 13:01:29.864825 11777 webserver.cc:460] Webserver started at http://127.21.235.126:42775/ using document root <none> and password file <none>
I20260809 13:01:29.865595 11777 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:29.865756 11777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:29.866111 11777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:29.869503 11777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "275b425a164f4059a3e67bedfca94efd"
format_stamp: "Formatted at 2026-08-09 13:01:29 on dist-test-slave-gp6n"
server_key: "c56451bfb27937a5dc72cbdbe282f1fb"
server_key_iv: "80ace34bea77ec17c0242372598567a8"
server_key_version: "encryptionkey@0"
I20260809 13:01:29.870324 11777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "275b425a164f4059a3e67bedfca94efd"
format_stamp: "Formatted at 2026-08-09 13:01:29 on dist-test-slave-gp6n"
server_key: "c56451bfb27937a5dc72cbdbe282f1fb"
server_key_iv: "80ace34bea77ec17c0242372598567a8"
server_key_version: "encryptionkey@0"
I20260809 13:01:29.875975 11777 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 13:01:29.879622 11795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:29.881021 11777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 13:01:29.881225 11777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "275b425a164f4059a3e67bedfca94efd"
format_stamp: "Formatted at 2026-08-09 13:01:29 on dist-test-slave-gp6n"
server_key: "c56451bfb27937a5dc72cbdbe282f1fb"
server_key_iv: "80ace34bea77ec17c0242372598567a8"
server_key_version: "encryptionkey@0"
I20260809 13:01:29.881454 11777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:29.912675 11777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:29.916874 11777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:29.917202 11777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:29.937366 11777 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:33125
I20260809 13:01:29.937378 11854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:33125 every 8 connection(s)
I20260809 13:01:29.938897 11777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:01:29.942929 11855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:29.946177 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 11777
I20260809 13:01:29.946362 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:01:29.946960 22445 external_mini_cluster.cc:1428] Setting key ef4e7b9598531d8ff658e1f1c8a8dbd1
I20260809 13:01:29.956223 11855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd: Bootstrap starting.
I20260809 13:01:29.960121 11855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:29.961555 11855 log.cc:824] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd: Log is configured to *not* fsync() on all Append() calls
Aug 09 13:01:29 dist-test-slave-gp6n krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:01:29.964826 11855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd: No bootstrap required, opened a new log
I20260809 13:01:29.971683 11859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:29.949047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35914 (local address 127.21.235.126:33125)
0809 13:01:29.949915 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:29.949928 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:29.949970 (+ 42us) server_negotiation.cc:407] Connection header received
0809 13:01:29.951750 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:29.951773 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:29.952185 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:29.952531 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:29.954325 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:29.955864 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:29.957667 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:29.958363 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:29.958568 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:29.963876 (+ 5308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:29.963885 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:29.963900 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:29.963940 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:29.967249 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:29.968057 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:29.968064 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:29.968074 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:29.968254 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:29.968669 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:29.968674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:29.968677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:29.969285 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:29.969666 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:29.970134 (+ 468us) server_negotiation.cc:299] Negotiation successful
0809 13:01:29.970629 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":428,"threads_started":1}
I20260809 13:01:29.973105 11855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "275b425a164f4059a3e67bedfca94efd" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33125 } }
I20260809 13:01:29.973531 11855 raft_consensus.cc:374] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:29.973657 11855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 275b425a164f4059a3e67bedfca94efd, State: Initialized, Role: FOLLOWER
I20260809 13:01:29.974236 11855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "275b425a164f4059a3e67bedfca94efd" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33125 } }
I20260809 13:01:29.974553 11855 raft_consensus.cc:388] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:01:29.974733 11855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:01:29.974905 11855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:29.977073 11855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "275b425a164f4059a3e67bedfca94efd" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33125 } }
I20260809 13:01:29.977517 11855 leader_election.cc:302] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 275b425a164f4059a3e67bedfca94efd; no voters:
I20260809 13:01:29.978117 11855 leader_election.cc:288] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:01:29.978233 11861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:29.978997 11861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd [term 1 LEADER]: Becoming Leader. State: Replica: 275b425a164f4059a3e67bedfca94efd, State: Running, Role: LEADER
I20260809 13:01:29.979581 11861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "275b425a164f4059a3e67bedfca94efd" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33125 } }
I20260809 13:01:29.980417 11855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:01:29.982969 11862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "275b425a164f4059a3e67bedfca94efd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "275b425a164f4059a3e67bedfca94efd" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33125 } } }
I20260809 13:01:29.982923 11863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 275b425a164f4059a3e67bedfca94efd. Latest consensus state: current_term: 1 leader_uuid: "275b425a164f4059a3e67bedfca94efd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "275b425a164f4059a3e67bedfca94efd" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 33125 } } }
I20260809 13:01:29.983505 11862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:29.983515 11863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:29.987370 11870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:01:29.993393 11870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:01:30.003338 11870 catalog_manager.cc:1347] Generated new cluster ID: 9443804741034c6e9939a61a9d5df157
I20260809 13:01:30.003504 11870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:01:30.022764 11870 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:01:30.024134 11870 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:01:30.036471 11870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd: Generated new TSK 0
I20260809 13:01:30.037470 11870 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
Entry for principal kudu/127.21.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab.
I20260809 13:01:30.130581 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:33125
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:35039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:30.487929 11885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:30.488247 11885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:30.488368 11885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:30.495857 11885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:30.495965 11885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:30.496078 11885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 13:01:30.505920 11885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:35039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.65.keytab
--principal=kudu/127.21.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:33125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:30.507246 11885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:30.508575 11885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:30.521946 11891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:30.521899 11892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:30.521982 11894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:30.523022 11885 server_base.cc:1034] running on GCE node
I20260809 13:01:30.523646 11885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:30.524570 11885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:30.525797 11885 hybrid_clock.cc:648] HybridClock initialized: now 1786280490525752 us; error 52 us; skew 500 ppm
I20260809 13:01:30.526125 11885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:30 dist-test-slave-gp6n krb5kdc[11763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280490, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:30.529332 11885 init.cc:377] Logged in from keytab as kudu/127.21.235.65@KRBTEST.COM (short username kudu)
I20260809 13:01:30.530748 11885 webserver.cc:460] Webserver started at http://127.21.235.65:45553/ using document root <none> and password file <none>
I20260809 13:01:30.531448 11885 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:30.531598 11885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:30.531909 11885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:30.534824 11885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "a99a88ae06114fa0b840ce88d2b0825d"
format_stamp: "Formatted at 2026-08-09 13:01:30 on dist-test-slave-gp6n"
server_key: "4b77099e9ca688909701625fcfe1e93f"
server_key_iv: "32a8c4c4cf871402716f062cf9d3075c"
server_key_version: "encryptionkey@0"
I20260809 13:01:30.535669 11885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "a99a88ae06114fa0b840ce88d2b0825d"
format_stamp: "Formatted at 2026-08-09 13:01:30 on dist-test-slave-gp6n"
server_key: "4b77099e9ca688909701625fcfe1e93f"
server_key_iv: "32a8c4c4cf871402716f062cf9d3075c"
server_key_version: "encryptionkey@0"
I20260809 13:01:30.541097 11885 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 13:01:30.544463 11902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:30.545717 11885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:01:30.545883 11885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "a99a88ae06114fa0b840ce88d2b0825d"
format_stamp: "Formatted at 2026-08-09 13:01:30 on dist-test-slave-gp6n"
server_key: "4b77099e9ca688909701625fcfe1e93f"
server_key_iv: "32a8c4c4cf871402716f062cf9d3075c"
server_key_version: "encryptionkey@0"
I20260809 13:01:30.546070 11885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:30.564086 11885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:30.568439 11885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:30.568732 11885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:30.569707 11885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:01:30.571075 11885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:01:30.571146 11885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:30.571295 11885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:01:30.571355 11885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:30.605342 11885 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:39741
I20260809 13:01:30.605397 12028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:39741 every 8 connection(s)
I20260809 13:01:30.606690 11885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 13:01:30.613579 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 11885
I20260809 13:01:30.613771 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 13:01:30.614240 22445 external_mini_cluster.cc:1428] Setting key 615d23b4b68ca2babd2b4875e5cbc315
Aug 09 13:01:30 dist-test-slave-gp6n krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280490, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.65@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:01:30.628947 12033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:30.609251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:50523 (local address 127.21.235.126:33125)
0809 13:01:30.609783 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:30.609790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:30.610759 (+ 969us) server_negotiation.cc:407] Connection header received
0809 13:01:30.612510 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:30.612514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:30.612586 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:30.612704 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:30.614269 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:30.615294 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:30.617304 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:30.618054 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:30.618208 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:30.621910 (+ 3702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:30.621919 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:30.621926 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:30.621971 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:30.625638 (+ 3667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:30.626360 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:30.626369 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:30.626373 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:30.626552 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:30.626960 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:30.626968 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:30.626973 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:30.627361 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:30.627506 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:30.628200 (+ 694us) server_negotiation.cc:299] Negotiation successful
0809 13:01:30.628466 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":239,"threads_started":1}
I20260809 13:01:30.629639 12035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:30.609806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33125 (local address 127.21.235.65:50523)
0809 13:01:30.610595 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:30.610640 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:30.612256 (+ 1616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:30.612901 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:30.612909 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:30.613436 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:01:30.614091 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:30.614110 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:30.615505 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:30.615509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:30.617060 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:30.617073 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:30.618272 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:30.618276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:30.618542 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:30.619354 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:01:30.619399 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:01:30.621693 (+ 2294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:01:30.625790 (+ 4097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:01:30.625797 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:01:30.625809 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:01:30.626241 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:01:30.626669 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:01:30.626673 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:01:30.626675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:01:30.626852 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:01:30.627500 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:01:30.627505 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:01:30.627820 (+ 315us) client_negotiation.cc:769] Sending connection context
0809 13:01:30.628048 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 13:01:30.628427 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":278,"threads_started":1}
I20260809 13:01:30.631402 12031 heartbeater.cc:347] Connected to a master server at 127.21.235.126:33125
I20260809 13:01:30.631727 12031 heartbeater.cc:464] Registering TS with master...
I20260809 13:01:30.632468 12031 heartbeater.cc:511] Master 127.21.235.126:33125 requested a full tablet report, sending...
I20260809 13:01:30.634864 11814 ts_manager.cc:194] Registered new tserver with Master: a99a88ae06114fa0b840ce88d2b0825d (127.21.235.65:39741)
I20260809 13:01:30.637045 11814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.65@KRBTEST.COM'} at 127.21.235.65:50523
WARNING: no policy specified for kudu/127.21.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
Entry for principal kudu/127.21.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab.
I20260809 13:01:30.683743 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:33125
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:35039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:31.029776 12038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:31.030099 12038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:31.030262 12038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:31.038674 12038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:31.038812 12038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:31.039000 12038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 13:01:31.049381 12038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:35039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.66.keytab
--principal=kudu/127.21.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:33125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:31.050946 12038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:31.052332 12038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:31.066390 12046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:31.066537 12044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:31.066531 12043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:31.067879 12038 server_base.cc:1034] running on GCE node
I20260809 13:01:31.068503 12038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:31.069528 12038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:31.070755 12038 hybrid_clock.cc:648] HybridClock initialized: now 1786280491070705 us; error 64 us; skew 500 ppm
I20260809 13:01:31.071096 12038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:31 dist-test-slave-gp6n krb5kdc[11763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280491, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:31.074554 12038 init.cc:377] Logged in from keytab as kudu/127.21.235.66@KRBTEST.COM (short username kudu)
I20260809 13:01:31.076135 12038 webserver.cc:460] Webserver started at http://127.21.235.66:40619/ using document root <none> and password file <none>
I20260809 13:01:31.076846 12038 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:31.077005 12038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:31.077332 12038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:31.080523 12038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "d591c9e05b8f42afbe26f38ae1322094"
format_stamp: "Formatted at 2026-08-09 13:01:31 on dist-test-slave-gp6n"
server_key: "aa506329c9766e7ec463d28975147ccd"
server_key_iv: "13535359a3522bfa6366b5f6d736cfdb"
server_key_version: "encryptionkey@0"
I20260809 13:01:31.081388 12038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "d591c9e05b8f42afbe26f38ae1322094"
format_stamp: "Formatted at 2026-08-09 13:01:31 on dist-test-slave-gp6n"
server_key: "aa506329c9766e7ec463d28975147ccd"
server_key_iv: "13535359a3522bfa6366b5f6d736cfdb"
server_key_version: "encryptionkey@0"
I20260809 13:01:31.087019 12038 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 13:01:31.090759 12053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:31.092044 12038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:01:31.092221 12038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "d591c9e05b8f42afbe26f38ae1322094"
format_stamp: "Formatted at 2026-08-09 13:01:31 on dist-test-slave-gp6n"
server_key: "aa506329c9766e7ec463d28975147ccd"
server_key_iv: "13535359a3522bfa6366b5f6d736cfdb"
server_key_version: "encryptionkey@0"
I20260809 13:01:31.092427 12038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:31.112890 12038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:31.117648 12038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:31.117972 12038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:31.119051 12038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:01:31.120515 12038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:01:31.120625 12038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:31.120779 12038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:01:31.120862 12038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:31.156140 12038 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:40111
I20260809 13:01:31.156205 12182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:40111 every 8 connection(s)
I20260809 13:01:31.157577 12038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 13:01:31.165241 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 12038
I20260809 13:01:31.165488 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 13:01:31.165952 22445 external_mini_cluster.cc:1428] Setting key 807a4903e35c4454ee49f8a35f3e56e7
Aug 09 13:01:31 dist-test-slave-gp6n krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280491, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.66@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:01:31.179188 12186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:31.160429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:35647 (local address 127.21.235.126:33125)
0809 13:01:31.160982 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:31.160986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:31.161826 (+ 840us) server_negotiation.cc:407] Connection header received
0809 13:01:31.163804 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:31.163810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:31.163907 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:31.164044 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:31.165799 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.166735 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.168688 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.169145 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.169301 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:31.173151 (+ 3850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:31.173157 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:31.173160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:31.173193 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:31.175928 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:31.176733 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:31.176745 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:31.176748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:31.176880 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:31.177339 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:31.177345 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:31.177348 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:31.177623 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:31.177784 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:31.178537 (+ 753us) server_negotiation.cc:299] Negotiation successful
0809 13:01:31.178767 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":293,"threads_started":1}
I20260809 13:01:31.180166 12187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:31.160907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33125 (local address 127.21.235.66:35647)
0809 13:01:31.161644 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:31.161693 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:31.163507 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:31.164305 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:31.164314 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:31.164908 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:01:31.165614 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:31.165636 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.166925 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.166931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:31.168517 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:31.168531 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.169419 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.169428 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:31.169666 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:31.170586 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:01:31.170612 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:01:31.172905 (+ 2293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:01:31.176078 (+ 3173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:01:31.176087 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:01:31.176105 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:01:31.176599 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:01:31.177005 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:01:31.177010 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:01:31.177014 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:01:31.177220 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:01:31.177757 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:01:31.177765 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:01:31.178132 (+ 367us) client_negotiation.cc:769] Sending connection context
0809 13:01:31.178441 (+ 309us) client_negotiation.cc:240] Negotiation successful
0809 13:01:31.179004 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":279,"threads_started":1}
I20260809 13:01:31.181820 12183 heartbeater.cc:347] Connected to a master server at 127.21.235.126:33125
I20260809 13:01:31.182159 12183 heartbeater.cc:464] Registering TS with master...
I20260809 13:01:31.183086 12183 heartbeater.cc:511] Master 127.21.235.126:33125 requested a full tablet report, sending...
I20260809 13:01:31.184751 11814 ts_manager.cc:194] Registered new tserver with Master: d591c9e05b8f42afbe26f38ae1322094 (127.21.235.66:40111)
I20260809 13:01:31.185696 11814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.66@KRBTEST.COM'} at 127.21.235.66:35647
WARNING: no policy specified for kudu/127.21.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
Entry for principal kudu/127.21.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab.
I20260809 13:01:31.235975 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:33125
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:35039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:31.578393 12191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:31.578809 12191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:31.578996 12191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:31.586923 12191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:31.587090 12191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:31.587303 12191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 13:01:31.597821 12191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:35039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.67.keytab
--principal=kudu/127.21.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:33125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:31.599251 12191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:31.600612 12191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:31.614717 12197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:31.614959 12198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:31.615015 12200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:31.615720 12191 server_base.cc:1034] running on GCE node
I20260809 13:01:31.616330 12191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:31.617267 12191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:31.618500 12191 hybrid_clock.cc:648] HybridClock initialized: now 1786280491618423 us; error 56 us; skew 500 ppm
I20260809 13:01:31.618829 12191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:31 dist-test-slave-gp6n krb5kdc[11763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280491, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:31.622017 12191 init.cc:377] Logged in from keytab as kudu/127.21.235.67@KRBTEST.COM (short username kudu)
I20260809 13:01:31.623566 12191 webserver.cc:460] Webserver started at http://127.21.235.67:41743/ using document root <none> and password file <none>
I20260809 13:01:31.624255 12191 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:31.624413 12191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:31.624738 12191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:31.627799 12191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "8a26100842074e1994faf163ad032456"
format_stamp: "Formatted at 2026-08-09 13:01:31 on dist-test-slave-gp6n"
server_key: "6686b8a5a42feef4a41751377adcbe56"
server_key_iv: "5b0408f3da70a6788b91446c1125aaff"
server_key_version: "encryptionkey@0"
I20260809 13:01:31.628599 12191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "8a26100842074e1994faf163ad032456"
format_stamp: "Formatted at 2026-08-09 13:01:31 on dist-test-slave-gp6n"
server_key: "6686b8a5a42feef4a41751377adcbe56"
server_key_iv: "5b0408f3da70a6788b91446c1125aaff"
server_key_version: "encryptionkey@0"
I20260809 13:01:31.633870 12191 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 13:01:31.637472 12208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:31.638943 12191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:01:31.639132 12191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "8a26100842074e1994faf163ad032456"
format_stamp: "Formatted at 2026-08-09 13:01:31 on dist-test-slave-gp6n"
server_key: "6686b8a5a42feef4a41751377adcbe56"
server_key_iv: "5b0408f3da70a6788b91446c1125aaff"
server_key_version: "encryptionkey@0"
I20260809 13:01:31.639351 12191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:31.640046 12031 heartbeater.cc:503] Master 127.21.235.126:33125 was elected leader, sending a full tablet report...
I20260809 13:01:31.652153 12191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:31.656162 12191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:31.656450 12191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:31.657402 12191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:01:31.658819 12191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:01:31.658919 12191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:31.659065 12191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:01:31.659139 12191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:31.692864 12191 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:44127
I20260809 13:01:31.692914 12334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:44127 every 8 connection(s)
I20260809 13:01:31.694175 12191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 13:01:31.696552 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 12191
I20260809 13:01:31.696759 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 13:01:31.697328 22445 external_mini_cluster.cc:1428] Setting key 4cac928f8e05c4de8e3d7b1d50f6947c
Aug 09 13:01:31 dist-test-slave-gp6n krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280491, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.67@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:01:31.715073 12337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:31.697162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:40533 (local address 127.21.235.126:33125)
0809 13:01:31.697673 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:31.697679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:31.698917 (+ 1238us) server_negotiation.cc:407] Connection header received
0809 13:01:31.700765 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:31.700770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:31.700850 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:31.700966 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:31.702499 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.703367 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.704892 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.705298 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.705399 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:31.708858 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:31.708863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:31.708866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:31.708905 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:31.711945 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:31.712670 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:31.712677 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:31.712681 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:31.712813 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:31.713247 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:31.713255 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:31.713259 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:31.713545 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:31.713711 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:31.714437 (+ 726us) server_negotiation.cc:299] Negotiation successful
0809 13:01:31.714665 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":263,"threads_started":1}
I20260809 13:01:31.716009 12338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:31.697819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:33125 (local address 127.21.235.67:40533)
0809 13:01:31.698747 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:31.698794 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:31.700498 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:31.701136 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:31.701144 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:31.701678 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:01:31.702258 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:31.702276 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.703512 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.703517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:31.704749 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:31.704766 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.705490 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.705494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:31.705682 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:31.706290 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:01:31.706328 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:01:31.708643 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:01:31.712087 (+ 3444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:01:31.712095 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:01:31.712112 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:01:31.712548 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:01:31.712929 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:01:31.712934 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:01:31.712938 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:01:31.713133 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:01:31.713675 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:01:31.713684 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:01:31.714039 (+ 355us) client_negotiation.cc:769] Sending connection context
0809 13:01:31.714316 (+ 277us) client_negotiation.cc:240] Negotiation successful
0809 13:01:31.714877 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":408,"threads_started":1}
I20260809 13:01:31.717312 12335 heartbeater.cc:347] Connected to a master server at 127.21.235.126:33125
I20260809 13:01:31.717594 12335 heartbeater.cc:464] Registering TS with master...
I20260809 13:01:31.718259 12335 heartbeater.cc:511] Master 127.21.235.126:33125 requested a full tablet report, sending...
I20260809 13:01:31.719693 11814 ts_manager.cc:194] Registered new tserver with Master: 8a26100842074e1994faf163ad032456 (127.21.235.67:44127)
I20260809 13:01:31.720526 11814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.235.67@KRBTEST.COM'} at 127.21.235.67:40533
I20260809 13:01:31.725395 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:01:31.747195 12337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:31.731996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58392 (local address 127.21.235.126:33125)
0809 13:01:31.732248 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:31.732257 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:31.732738 (+ 481us) server_negotiation.cc:407] Connection header received
0809 13:01:31.732885 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:31.732889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:31.733025 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:31.733169 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:31.734688 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.735480 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.737249 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.737648 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.737748 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:31.741017 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:31.741022 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:31.741026 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:31.741058 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:31.744042 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:31.745142 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:31.745147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:31.745149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:31.745274 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:31.745659 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:31.745663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:31.745665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:31.746003 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:31.746187 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:31.746611 (+ 424us) server_negotiation.cc:299] Negotiation successful
0809 13:01:31.746796 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 13:01:31.751626 11814 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:01:31.758623 11814 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:01:31.784269 12352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:31.773217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39854 (local address 127.21.235.65:39741)
0809 13:01:31.775431 (+ 2214us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:31.775445 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:31.775482 (+ 37us) server_negotiation.cc:407] Connection header received
0809 13:01:31.775605 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:31.775618 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:31.775932 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:31.776357 (+ 425us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:31.777150 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.778979 (+ 1829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.781642 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.782995 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.783154 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:31.783402 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:31.783737 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 13:01:31.783907 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1972,"thread_start_us":1793,"threads_started":1}
I20260809 13:01:31.784371 12350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:31.772566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:39741 (local address 127.0.0.1:39854)
0809 13:01:31.773695 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:31.773718 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:31.773857 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:31.776198 (+ 2341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:31.776206 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:31.776242 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:31.776973 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:31.776987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.779129 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.779139 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:31.781475 (+ 2336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:31.781488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.783196 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.783201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:31.783451 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:31.783517 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 13:01:31.783671 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 13:01:31.783885 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":958,"mutex_wait_us":355,"thread_start_us":831,"threads_started":1}
I20260809 13:01:31.784425 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:31.771669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:44127 (local address 127.0.0.1:53300)
0809 13:01:31.772501 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:31.772556 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:31.772782 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:31.777429 (+ 4647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:31.777433 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:31.777460 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:31.777829 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:31.777843 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.779599 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.779606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:31.781593 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:31.781607 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.783394 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.783403 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:31.783585 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:31.783608 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 13:01:31.783715 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 13:01:31.783893 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":59,"thread_start_us":299,"threads_started":1}
I20260809 13:01:31.785012 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:31.772793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:40111 (local address 127.0.0.1:45254)
0809 13:01:31.774462 (+ 1669us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:31.774479 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:31.774601 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:31.776172 (+ 1571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:31.776179 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:31.776207 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:31.776654 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:31.776667 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.778648 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.778652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:31.781306 (+ 2654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:31.781322 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.783565 (+ 2243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.783571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:31.783758 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:31.783793 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 13:01:31.783868 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 13:01:31.783980 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1491,"mutex_wait_us":32,"thread_start_us":1333,"threads_started":1}
I20260809 13:01:31.785544 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:31.771911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53300 (local address 127.21.235.67:44127)
0809 13:01:31.776867 (+ 4956us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:31.776875 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:31.776906 (+ 31us) server_negotiation.cc:407] Connection header received
0809 13:01:31.777016 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:31.777025 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:31.777229 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:31.777414 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:31.778031 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.779460 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.781777 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.783161 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.784822 (+ 1661us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:31.785101 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:31.785172 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 13:01:31.785326 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4777,"thread_start_us":4619,"threads_started":1}
I20260809 13:01:31.785534 12353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:31.773271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45254 (local address 127.21.235.66:40111)
0809 13:01:31.775692 (+ 2421us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:31.775699 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:31.775721 (+ 22us) server_negotiation.cc:407] Connection header received
0809 13:01:31.775794 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:31.775799 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:31.775970 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:31.776168 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:31.776889 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.778532 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.781470 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.783041 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.783179 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:31.783421 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:31.784132 (+ 711us) server_negotiation.cc:299] Negotiation successful
0809 13:01:31.784301 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2267,"thread_start_us":2160,"threads_started":1}
I20260809 13:01:31.788744 11956 tablet_service.cc:1467] Processing CreateTablet for tablet 58ee2bb909d24400a7118fe61b2802ac (DEFAULT_TABLE table=test-table [id=00f075c84b4449b396166f1b130b6ea9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:01:31.789083 12261 tablet_service.cc:1467] Processing CreateTablet for tablet 58ee2bb909d24400a7118fe61b2802ac (DEFAULT_TABLE table=test-table [id=00f075c84b4449b396166f1b130b6ea9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:01:31.789083 12107 tablet_service.cc:1467] Processing CreateTablet for tablet 58ee2bb909d24400a7118fe61b2802ac (DEFAULT_TABLE table=test-table [id=00f075c84b4449b396166f1b130b6ea9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:01:31.790074 11956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58ee2bb909d24400a7118fe61b2802ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:31.790469 12261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58ee2bb909d24400a7118fe61b2802ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:31.790488 12107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58ee2bb909d24400a7118fe61b2802ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:31.801939 12354 tablet_bootstrap.cc:492] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d: Bootstrap starting.
I20260809 13:01:31.804942 12355 tablet_bootstrap.cc:492] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094: Bootstrap starting.
I20260809 13:01:31.805462 12354 tablet_bootstrap.cc:654] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:31.806877 12354 log.cc:824] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:31.807116 12356 tablet_bootstrap.cc:492] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456: Bootstrap starting.
I20260809 13:01:31.808964 12355 tablet_bootstrap.cc:654] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:31.810305 12355 log.cc:824] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:31.810830 12356 tablet_bootstrap.cc:654] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:31.811139 12354 tablet_bootstrap.cc:492] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d: No bootstrap required, opened a new log
I20260809 13:01:31.811412 12354 ts_tablet_manager.cc:1397] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 13:01:31.812024 12356 log.cc:824] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:31.813416 12355 tablet_bootstrap.cc:492] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094: No bootstrap required, opened a new log
I20260809 13:01:31.813664 12355 ts_tablet_manager.cc:1397] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 13:01:31.814383 12356 tablet_bootstrap.cc:492] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456: No bootstrap required, opened a new log
I20260809 13:01:31.814673 12356 ts_tablet_manager.cc:1397] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260809 13:01:31.821673 12356 raft_consensus.cc:348] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a26100842074e1994faf163ad032456" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44127 } } peers { permanent_uuid: "d591c9e05b8f42afbe26f38ae1322094" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40111 } } peers { permanent_uuid: "a99a88ae06114fa0b840ce88d2b0825d" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39741 } }
I20260809 13:01:31.822045 12356 raft_consensus.cc:374] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:31.822153 12356 raft_consensus.cc:729] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a26100842074e1994faf163ad032456, State: Initialized, Role: FOLLOWER
I20260809 13:01:31.821980 12354 raft_consensus.cc:348] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a26100842074e1994faf163ad032456" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44127 } } peers { permanent_uuid: "d591c9e05b8f42afbe26f38ae1322094" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40111 } } peers { permanent_uuid: "a99a88ae06114fa0b840ce88d2b0825d" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39741 } }
I20260809 13:01:31.822427 12354 raft_consensus.cc:374] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:31.822556 12354 raft_consensus.cc:729] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a99a88ae06114fa0b840ce88d2b0825d, State: Initialized, Role: FOLLOWER
I20260809 13:01:31.822937 12356 consensus_queue.cc:260] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a26100842074e1994faf163ad032456" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44127 } } peers { permanent_uuid: "d591c9e05b8f42afbe26f38ae1322094" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40111 } } peers { permanent_uuid: "a99a88ae06114fa0b840ce88d2b0825d" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39741 } }
I20260809 13:01:31.823148 12354 consensus_queue.cc:260] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a26100842074e1994faf163ad032456" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44127 } } peers { permanent_uuid: "d591c9e05b8f42afbe26f38ae1322094" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40111 } } peers { permanent_uuid: "a99a88ae06114fa0b840ce88d2b0825d" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39741 } }
I20260809 13:01:31.824182 12335 heartbeater.cc:503] Master 127.21.235.126:33125 was elected leader, sending a full tablet report...
I20260809 13:01:31.824288 12355 raft_consensus.cc:348] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a26100842074e1994faf163ad032456" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44127 } } peers { permanent_uuid: "d591c9e05b8f42afbe26f38ae1322094" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40111 } } peers { permanent_uuid: "a99a88ae06114fa0b840ce88d2b0825d" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39741 } }
I20260809 13:01:31.824900 12355 raft_consensus.cc:374] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:31.824920 12356 ts_tablet_manager.cc:1428] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456: Time spent starting tablet: real 0.010s user 0.000s sys 0.009s
I20260809 13:01:31.825079 12355 raft_consensus.cc:729] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d591c9e05b8f42afbe26f38ae1322094, State: Initialized, Role: FOLLOWER
I20260809 13:01:31.825846 12355 consensus_queue.cc:260] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a26100842074e1994faf163ad032456" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44127 } } peers { permanent_uuid: "d591c9e05b8f42afbe26f38ae1322094" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40111 } } peers { permanent_uuid: "a99a88ae06114fa0b840ce88d2b0825d" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39741 } }
I20260809 13:01:31.826797 12354 ts_tablet_manager.cc:1428] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
I20260809 13:01:31.827824 12183 heartbeater.cc:503] Master 127.21.235.126:33125 was elected leader, sending a full tablet report...
I20260809 13:01:31.828979 12355 ts_tablet_manager.cc:1428] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
W20260809 13:01:31.859237 12032 tablet.cc:2369] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:01:31.879858 12363 raft_consensus.cc:482] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:01:31.880182 12363 raft_consensus.cc:504] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a26100842074e1994faf163ad032456" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44127 } } peers { permanent_uuid: "d591c9e05b8f42afbe26f38ae1322094" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40111 } } peers { permanent_uuid: "a99a88ae06114fa0b840ce88d2b0825d" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39741 } }
I20260809 13:01:31.882254 12363 leader_election.cc:288] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a26100842074e1994faf163ad032456 (127.21.235.67:44127), d591c9e05b8f42afbe26f38ae1322094 (127.21.235.66:40111)
I20260809 13:01:31.890250 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:31.882656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:37989 (local address 127.21.235.67:44127)
0809 13:01:31.882856 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:31.882861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:31.883467 (+ 606us) server_negotiation.cc:407] Connection header received
0809 13:01:31.883724 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:31.883735 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:31.883838 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:31.883916 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:31.884752 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.885658 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.888203 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.889417 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.889589 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:31.889670 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:31.889907 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 13:01:31.890037 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 13:01:31.890480 12365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:31.882646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:44127 (local address 127.21.235.65:37989)
0809 13:01:31.883341 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:31.883369 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:31.883516 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:31.884108 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:31.884112 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:31.884145 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:31.884598 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:31.884608 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.885786 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.885792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:31.888008 (+ 2216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:31.888025 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.889621 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.889625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:31.889741 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:31.889765 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 13:01:31.889842 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 13:01:31.890008 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"spinlock_wait_cycles":110848,"thread_start_us":382,"threads_started":1}
I20260809 13:01:31.890890 12353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:31.883730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:52511 (local address 127.21.235.66:40111)
0809 13:01:31.883996 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:31.884003 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:31.884252 (+ 249us) server_negotiation.cc:407] Connection header received
0809 13:01:31.884523 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:31.884528 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:31.884595 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:31.884704 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:31.885360 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.886197 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.888544 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.889809 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.889975 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:31.890041 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:31.890490 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 13:01:31.890639 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260809 13:01:31.891568 12366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:31.883494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:40111 (local address 127.21.235.65:52511)
0809 13:01:31.884131 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:31.884146 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:31.884353 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:31.884721 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:31.884726 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:31.884752 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:31.885157 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:31.885171 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.886432 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.886438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:31.888400 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:31.888411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.890051 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.890056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:31.890240 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:31.890270 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 13:01:31.891092 (+ 822us) client_negotiation.cc:240] Negotiation successful
0809 13:01:31.891215 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":3,"thread_start_us":300,"threads_started":1}
I20260809 13:01:31.891491 12284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58ee2bb909d24400a7118fe61b2802ac" candidate_uuid: "a99a88ae06114fa0b840ce88d2b0825d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a26100842074e1994faf163ad032456" is_pre_election: true
I20260809 13:01:31.892161 12284 raft_consensus.cc:2454] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a99a88ae06114fa0b840ce88d2b0825d in term 0.
I20260809 13:01:31.892663 12129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58ee2bb909d24400a7118fe61b2802ac" candidate_uuid: "a99a88ae06114fa0b840ce88d2b0825d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d591c9e05b8f42afbe26f38ae1322094" is_pre_election: true
I20260809 13:01:31.892860 11907 leader_election.cc:302] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a26100842074e1994faf163ad032456, a99a88ae06114fa0b840ce88d2b0825d; no voters:
I20260809 13:01:31.893200 12129 raft_consensus.cc:2454] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a99a88ae06114fa0b840ce88d2b0825d in term 0.
I20260809 13:01:31.893332 12363 raft_consensus.cc:2781] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:01:31.893479 12363 raft_consensus.cc:482] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:01:31.893595 12363 raft_consensus.cc:3037] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:31.895913 12363 raft_consensus.cc:504] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a26100842074e1994faf163ad032456" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44127 } } peers { permanent_uuid: "d591c9e05b8f42afbe26f38ae1322094" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40111 } } peers { permanent_uuid: "a99a88ae06114fa0b840ce88d2b0825d" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39741 } }
I20260809 13:01:31.896503 12363 leader_election.cc:288] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [CANDIDATE]: Term 1 election: Requested vote from peers 8a26100842074e1994faf163ad032456 (127.21.235.67:44127), d591c9e05b8f42afbe26f38ae1322094 (127.21.235.66:40111)
I20260809 13:01:31.897145 12284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58ee2bb909d24400a7118fe61b2802ac" candidate_uuid: "a99a88ae06114fa0b840ce88d2b0825d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a26100842074e1994faf163ad032456"
I20260809 13:01:31.897190 12129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58ee2bb909d24400a7118fe61b2802ac" candidate_uuid: "a99a88ae06114fa0b840ce88d2b0825d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d591c9e05b8f42afbe26f38ae1322094"
I20260809 13:01:31.897388 12284 raft_consensus.cc:3037] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:31.897449 12129 raft_consensus.cc:3037] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:31.899822 12129 raft_consensus.cc:2454] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a99a88ae06114fa0b840ce88d2b0825d in term 1.
I20260809 13:01:31.900002 12284 raft_consensus.cc:2454] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a99a88ae06114fa0b840ce88d2b0825d in term 1.
I20260809 13:01:31.900246 11907 leader_election.cc:302] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a99a88ae06114fa0b840ce88d2b0825d, d591c9e05b8f42afbe26f38ae1322094; no voters:
I20260809 13:01:31.900596 12363 raft_consensus.cc:2781] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:31.901118 12363 raft_consensus.cc:686] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [term 1 LEADER]: Becoming Leader. State: Replica: a99a88ae06114fa0b840ce88d2b0825d, State: Running, Role: LEADER
I20260809 13:01:31.901762 12363 consensus_queue.cc:237] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a26100842074e1994faf163ad032456" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44127 } } peers { permanent_uuid: "d591c9e05b8f42afbe26f38ae1322094" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40111 } } peers { permanent_uuid: "a99a88ae06114fa0b840ce88d2b0825d" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39741 } }
I20260809 13:01:31.906697 11812 catalog_manager.cc:5495] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d reported cstate change: term changed from 0 to 1, leader changed from <none> to a99a88ae06114fa0b840ce88d2b0825d (127.21.235.65). New cstate: current_term: 1 leader_uuid: "a99a88ae06114fa0b840ce88d2b0825d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a26100842074e1994faf163ad032456" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d591c9e05b8f42afbe26f38ae1322094" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a99a88ae06114fa0b840ce88d2b0825d" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 39741 } health_report { overall_health: HEALTHY } } }
W20260809 13:01:31.909746 12184 tablet.cc:2369] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:01:31.935115 12352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:31.929839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39856 (local address 127.21.235.65:39741)
0809 13:01:31.930030 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:31.930034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:31.930051 (+ 17us) server_negotiation.cc:407] Connection header received
0809 13:01:31.930157 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:31.930160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:31.930238 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:31.930375 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:01:31.930843 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.931595 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.932977 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.933378 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.933471 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:31.933916 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:01:31.934533 (+ 617us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:01:31.934650 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:31.934739 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 13:01:31.934822 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 13:01:31.943684 12129 raft_consensus.cc:1261] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094 [term 1 FOLLOWER]: Refusing update from remote peer a99a88ae06114fa0b840ce88d2b0825d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:01:31.943693 12284 raft_consensus.cc:1261] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456 [term 1 FOLLOWER]: Refusing update from remote peer a99a88ae06114fa0b840ce88d2b0825d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:01:31.944784 12363 consensus_queue.cc:1035] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d591c9e05b8f42afbe26f38ae1322094" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 40111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:01:31.945467 12368 consensus_queue.cc:1035] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a26100842074e1994faf163ad032456" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 44127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 13:01:31.945914 12336 tablet.cc:2369] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:01:31.963491 12371 mvcc.cc:205] Tried to move back new op lower bound from 7316604894990262272 to 7316604894834184192. Current Snapshot: MvccSnapshot[applied={T|T < 7316604894990262272}]
I20260809 13:01:31.963886 12372 mvcc.cc:205] Tried to move back new op lower bound from 7316604894990262272 to 7316604894834184192. Current Snapshot: MvccSnapshot[applied={T|T < 7316604894990262272}]
I20260809 13:01:31.976568 11812 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:01:31.976887 11812 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:01:31.980752 11812 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 275b425a164f4059a3e67bedfca94efd: Sending DeleteTablet for 3 replicas of tablet 58ee2bb909d24400a7118fe61b2802ac
I20260809 13:01:31.982059 11956 tablet_service.cc:1514] Processing DeleteTablet for tablet 58ee2bb909d24400a7118fe61b2802ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:01:31 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:39854
I20260809 13:01:31.982054 12107 tablet_service.cc:1514] Processing DeleteTablet for tablet 58ee2bb909d24400a7118fe61b2802ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:01:31 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:45254
I20260809 13:01:31.982251 12261 tablet_service.cc:1514] Processing DeleteTablet for tablet 58ee2bb909d24400a7118fe61b2802ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:01:31 UTC) from {username='kudu', principal='kudu/127.21.235.126@KRBTEST.COM'} at 127.0.0.1:53300
I20260809 13:01:31.985826 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 11885
I20260809 13:01:31.993844 12383 tablet_replica.cc:330] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094: stopping tablet replica
I20260809 13:01:31.994031 12382 tablet_replica.cc:330] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456: stopping tablet replica
I20260809 13:01:31.994513 12382 raft_consensus.cc:2229] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:01:31.994514 12383 raft_consensus.cc:2229] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:01:31.995061 12383 raft_consensus.cc:2258] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:01:31.995061 12382 raft_consensus.cc:2258] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:01:31.995690 12381 tablet_replica.cc:330] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d: stopping tablet replica
I20260809 13:01:31.997717 12382 ts_tablet_manager.cc:1905] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:01:31.997808 12383 ts_tablet_manager.cc:1905] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:01:31.998296 12381 raft_consensus.cc:2229] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:01:31.999120 12381 raft_consensus.cc:2258] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 09 13:01:32 dist-test-slave-gp6n krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.65@KRBTEST.COM
I20260809 13:01:32.005160 12382 ts_tablet_manager.cc:1918] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:01:32.005379 12382 log.cc:1196] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/58ee2bb909d24400a7118fe61b2802ac
I20260809 13:01:32.005966 12382 ts_tablet_manager.cc:1939] T 58ee2bb909d24400a7118fe61b2802ac P 8a26100842074e1994faf163ad032456: Deleting consensus metadata
I20260809 13:01:32.006244 12381 ts_tablet_manager.cc:1905] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:01:32.006950 12383 ts_tablet_manager.cc:1918] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:01:32.007182 12383 log.cc:1196] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/58ee2bb909d24400a7118fe61b2802ac
I20260809 13:01:32.008062 11799 catalog_manager.cc:4841] TS 8a26100842074e1994faf163ad032456 (127.21.235.67:44127): tablet 58ee2bb909d24400a7118fe61b2802ac (table test-table [id=00f075c84b4449b396166f1b130b6ea9]) successfully deleted
I20260809 13:01:32.008448 12383 ts_tablet_manager.cc:1939] T 58ee2bb909d24400a7118fe61b2802ac P d591c9e05b8f42afbe26f38ae1322094: Deleting consensus metadata
I20260809 13:01:32.009976 11799 catalog_manager.cc:4841] TS d591c9e05b8f42afbe26f38ae1322094 (127.21.235.66:40111): tablet 58ee2bb909d24400a7118fe61b2802ac (table test-table [id=00f075c84b4449b396166f1b130b6ea9]) successfully deleted
I20260809 13:01:32.014298 12381 ts_tablet_manager.cc:1918] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:01:32.014504 12381 log.cc:1196] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280163688078-22445-0/minicluster-data/ts-0/wal/wals/58ee2bb909d24400a7118fe61b2802ac
I20260809 13:01:32.015081 12381 ts_tablet_manager.cc:1939] T 58ee2bb909d24400a7118fe61b2802ac P a99a88ae06114fa0b840ce88d2b0825d: Deleting consensus metadata
I20260809 13:01:32.016970 11798 catalog_manager.cc:4841] TS a99a88ae06114fa0b840ce88d2b0825d (127.21.235.65:39741): tablet 58ee2bb909d24400a7118fe61b2802ac (table test-table [id=00f075c84b4449b396166f1b130b6ea9]) successfully deleted
I20260809 13:01:32.017916 12352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:31.987889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39868 (local address 127.21.235.65:39741)
0809 13:01:31.988062 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:31.988069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:31.991594 (+ 3525us) server_negotiation.cc:407] Connection header received
0809 13:01:31.991821 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:31.991827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:31.991918 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:31.992013 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:31.995706 (+ 3693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.996943 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.998956 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:31.999674 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:31.999787 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:32.011474 (+ 11687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:32.011488 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:32.011507 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:32.011559 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:32.014984 (+ 3425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:32.015742 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:32.015748 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:32.015751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:32.015907 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:32.016344 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:32.016348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:32.016350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:32.016782 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:32.017087 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:32.017429 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 13:01:32.017609 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 13:01:32.018659 12024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:32.118815 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 11885
I20260809 13:01:32.136926 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 12038
Aug 09 13:01:32 dist-test-slave-gp6n krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.66@KRBTEST.COM
I20260809 13:01:32.152561 12353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:32.137771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45264 (local address 127.21.235.66:40111)
0809 13:01:32.137961 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:32.137966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:32.137988 (+ 22us) server_negotiation.cc:407] Connection header received
0809 13:01:32.138163 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:32.138167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:32.138232 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:32.138317 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:32.139779 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:32.140513 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:32.141773 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:32.142161 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:32.142261 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:32.146791 (+ 4530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:32.146806 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:32.146821 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:32.146864 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:32.149935 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:32.150669 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:32.150677 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:32.150680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:32.150811 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:32.151145 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:32.151148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:32.151150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:32.151581 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:32.151798 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:32.152020 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 13:01:32.152183 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 13:01:32.153285 12177 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:32.247704 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 12038
I20260809 13:01:32.269189 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 12191
Aug 09 13:01:32 dist-test-slave-gp6n krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.67@KRBTEST.COM
I20260809 13:01:32.285872 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:32.270095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53304 (local address 127.21.235.67:44127)
0809 13:01:32.270258 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:32.270263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:32.270464 (+ 201us) server_negotiation.cc:407] Connection header received
0809 13:01:32.270845 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:32.270859 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:32.270965 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:32.271086 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:32.272613 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:32.273428 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:32.274805 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:32.275234 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:32.275345 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:32.279964 (+ 4619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:32.279979 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:32.279995 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:32.280035 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:32.282905 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:32.283580 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:32.283587 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:32.283589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:32.283715 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:32.284056 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:32.284060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:32.284062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:32.284411 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:32.284633 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:32.285109 (+ 476us) server_negotiation.cc:299] Negotiation successful
0809 13:01:32.285384 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 13:01:32.286605 12330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:32.376672 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 12191
I20260809 13:01:32.396109 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 11777
I20260809 13:01:32.397222 11848 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:32.511111 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 11777
2026-08-09T13:01:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4548 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T13:01:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:01:32Z Disabled control of system clock
I20260809 13:01:32.601429 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:36127
--webserver_interface=127.21.235.126
--webserver_port=0
--builtin_ntp_servers=127.21.235.84:33839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:36127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 13:01:32.958752 12392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:32.959055 12392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:32.959144 12392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:32.966487 12392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:01:32.966591 12392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:32.966650 12392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:32.966746 12392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:01:32.966820 12392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:01:32.976346 12392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:36127
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:36127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:32.977635 12392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:32.979944 12392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:32.994509 12400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:32.994457 12398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:32.994485 12397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:33.000373 12392 server_base.cc:1034] running on GCE node
I20260809 13:01:33.001317 12392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:33.002954 12392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:33.004228 12392 hybrid_clock.cc:648] HybridClock initialized: now 1786280493004157 us; error 64 us; skew 500 ppm
I20260809 13:01:33.004652 12392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:01:33.007157 12392 webserver.cc:460] Webserver started at http://127.21.235.126:41081/ using document root <none> and password file <none>
I20260809 13:01:33.007869 12392 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:33.008004 12392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:33.008340 12392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:33.011404 12392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "c5a28ce35fda4db69fd4922dace2eb39"
format_stamp: "Formatted at 2026-08-09 13:01:33 on dist-test-slave-gp6n"
server_key: "51d04dda38cc825c68027b2d2b60df4c"
server_key_iv: "3141c620b3f9c89433a00745e5c14547"
server_key_version: "encryptionkey@0"
I20260809 13:01:33.012221 12392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "c5a28ce35fda4db69fd4922dace2eb39"
format_stamp: "Formatted at 2026-08-09 13:01:33 on dist-test-slave-gp6n"
server_key: "51d04dda38cc825c68027b2d2b60df4c"
server_key_iv: "3141c620b3f9c89433a00745e5c14547"
server_key_version: "encryptionkey@0"
I20260809 13:01:33.017438 12392 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 13:01:33.020991 12407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:33.022377 12392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:01:33.022639 12392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "c5a28ce35fda4db69fd4922dace2eb39"
format_stamp: "Formatted at 2026-08-09 13:01:33 on dist-test-slave-gp6n"
server_key: "51d04dda38cc825c68027b2d2b60df4c"
server_key_iv: "3141c620b3f9c89433a00745e5c14547"
server_key_version: "encryptionkey@0"
I20260809 13:01:33.022854 12392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:33.049296 12392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:33.050241 12392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:33.050599 12392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:33.070045 12392 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:36127
I20260809 13:01:33.070082 12463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:36127 every 8 connection(s)
I20260809 13:01:33.071460 12392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:01:33.075277 12464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:33.077461 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 12392
I20260809 13:01:33.077642 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:01:33.078194 22445 external_mini_cluster.cc:1428] Setting key 7bfa67f012e6a87642285107014af566
I20260809 13:01:33.087639 12464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39: Bootstrap starting.
I20260809 13:01:33.091176 12464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:33.092224 12464 log.cc:824] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:33.095283 12464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39: No bootstrap required, opened a new log
I20260809 13:01:33.097215 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:33.080448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59614 (local address 127.21.235.126:36127)
0809 13:01:33.081508 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:33.081528 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:33.081569 (+ 41us) server_negotiation.cc:407] Connection header received
0809 13:01:33.083402 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:33.083425 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:33.083901 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:33.084311 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:33.084924 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:33.086310 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:33.087753 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:33.088518 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:33.088731 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:33.094808 (+ 6077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:33.094819 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:33.094834 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:33.095427 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:33.095529 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:33.095553 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:33.095704 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:33.095931 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 13:01:33.096248 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":427,"threads_started":1}
I20260809 13:01:33.103758 12464 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a28ce35fda4db69fd4922dace2eb39" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36127 } }
I20260809 13:01:33.104146 12464 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:33.104256 12464 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5a28ce35fda4db69fd4922dace2eb39, State: Initialized, Role: FOLLOWER
I20260809 13:01:33.104833 12464 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a28ce35fda4db69fd4922dace2eb39" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36127 } }
I20260809 13:01:33.105091 12464 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:01:33.105232 12464 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:01:33.105423 12464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:33.107790 12464 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a28ce35fda4db69fd4922dace2eb39" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36127 } }
I20260809 13:01:33.108289 12464 leader_election.cc:302] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5a28ce35fda4db69fd4922dace2eb39; no voters:
I20260809 13:01:33.108911 12464 leader_election.cc:288] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:01:33.109083 12469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:33.109879 12469 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39 [term 1 LEADER]: Becoming Leader. State: Replica: c5a28ce35fda4db69fd4922dace2eb39, State: Running, Role: LEADER
I20260809 13:01:33.110484 12469 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a28ce35fda4db69fd4922dace2eb39" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36127 } }
I20260809 13:01:33.111122 12464 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:01:33.113999 12471 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5a28ce35fda4db69fd4922dace2eb39. Latest consensus state: current_term: 1 leader_uuid: "c5a28ce35fda4db69fd4922dace2eb39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a28ce35fda4db69fd4922dace2eb39" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36127 } } }
I20260809 13:01:33.114140 12470 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5a28ce35fda4db69fd4922dace2eb39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a28ce35fda4db69fd4922dace2eb39" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 36127 } } }
I20260809 13:01:33.114470 12471 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:33.114531 12470 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:33.118790 12480 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:01:33.123193 12480 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:01:33.131541 12480 catalog_manager.cc:1347] Generated new cluster ID: ff3f107ed6aa4668aed2f05f86e3e764
I20260809 13:01:33.131666 12480 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:01:33.166704 12480 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:01:33.168561 12480 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:01:33.177461 12480 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39: Generated new TSK 0
I20260809 13:01:33.178539 12480 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:01:33.196952 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.65:0
--local_ip_for_outbound_sockets=127.21.235.65
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:36127
--builtin_ntp_servers=127.21.235.84:33839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 13:01:33.542773 12489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:33.543084 12489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:33.543255 12489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:33.550266 12489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:33.550406 12489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:33.550602 12489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.65
I20260809 13:01:33.560326 12489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.235.65
--webserver_port=0
--tserver_master_addrs=127.21.235.126:36127
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.65
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:33.561625 12489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:33.563424 12489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:33.576474 12494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:33.576443 12495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:33.576613 12497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:33.577203 12489 server_base.cc:1034] running on GCE node
I20260809 13:01:33.577822 12489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:33.578853 12489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:33.580116 12489 hybrid_clock.cc:648] HybridClock initialized: now 1786280493580056 us; error 60 us; skew 500 ppm
I20260809 13:01:33.580466 12489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:01:33.582870 12489 webserver.cc:460] Webserver started at http://127.21.235.65:34737/ using document root <none> and password file <none>
I20260809 13:01:33.583585 12489 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:33.583721 12489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:33.584055 12489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:33.587008 12489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-0/data/instance:
uuid: "0c1f5f4b622641b29983ceb22d2f3016"
format_stamp: "Formatted at 2026-08-09 13:01:33 on dist-test-slave-gp6n"
server_key: "65167a4de65c8d8dafb621304c616a3a"
server_key_iv: "e1ff3fdf2989618c3493c62f2354ab98"
server_key_version: "encryptionkey@0"
I20260809 13:01:33.587841 12489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance:
uuid: "0c1f5f4b622641b29983ceb22d2f3016"
format_stamp: "Formatted at 2026-08-09 13:01:33 on dist-test-slave-gp6n"
server_key: "65167a4de65c8d8dafb621304c616a3a"
server_key_iv: "e1ff3fdf2989618c3493c62f2354ab98"
server_key_version: "encryptionkey@0"
I20260809 13:01:33.593173 12489 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:01:33.596817 12503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:33.598228 12489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 13:01:33.598523 12489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-0/wal
uuid: "0c1f5f4b622641b29983ceb22d2f3016"
format_stamp: "Formatted at 2026-08-09 13:01:33 on dist-test-slave-gp6n"
server_key: "65167a4de65c8d8dafb621304c616a3a"
server_key_iv: "e1ff3fdf2989618c3493c62f2354ab98"
server_key_version: "encryptionkey@0"
I20260809 13:01:33.598794 12489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:33.620895 12489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:33.621793 12489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:33.622139 12489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:33.623304 12489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:01:33.624958 12489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:01:33.625119 12489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:33.625260 12489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:01:33.625362 12489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:33.662750 12489 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.65:46609
I20260809 13:01:33.662859 12631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.65:46609 every 8 connection(s)
I20260809 13:01:33.664186 12489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-0/data/info.pb
I20260809 13:01:33.669582 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 12489
I20260809 13:01:33.669772 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-0/wal/instance
I20260809 13:01:33.670337 22445 external_mini_cluster.cc:1428] Setting key 4f3c5067cc76a7a7859c0b1a664b4010
I20260809 13:01:33.673774 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.66:0
--local_ip_for_outbound_sockets=127.21.235.66
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:36127
--builtin_ntp_servers=127.21.235.84:33839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 13:01:33.679556 12634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:33.667024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:36557 (local address 127.21.235.126:36127)
0809 13:01:33.667483 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:33.667488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:33.668818 (+ 1330us) server_negotiation.cc:407] Connection header received
0809 13:01:33.671306 (+ 2488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:33.671314 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:33.671425 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:33.671562 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:33.672887 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:33.674177 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:33.675894 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:33.676768 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:33.676935 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:33.678506 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:33.678515 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:33.678524 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:33.678554 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:33.678614 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:33.678636 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:33.678747 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:33.678962 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 13:01:33.679087 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":232,"threads_started":1}
I20260809 13:01:33.680473 12635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:33.667702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:36127 (local address 127.21.235.65:36557)
0809 13:01:33.668624 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:33.668666 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:33.671006 (+ 2340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:33.671804 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:33.671812 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:33.671908 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:01:33.672691 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:33.672711 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:33.674355 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:33.674359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:33.675731 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:33.675743 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:33.677010 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:33.677020 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:33.677381 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:33.678135 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:01:33.678138 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:01:33.678181 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:01:33.678266 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:01:33.678742 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:01:33.678751 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:01:33.678758 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 13:01:33.678915 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 13:01:33.679228 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":288,"threads_started":1}
I20260809 13:01:33.682188 12632 heartbeater.cc:347] Connected to a master server at 127.21.235.126:36127
I20260809 13:01:33.682606 12632 heartbeater.cc:464] Registering TS with master...
I20260809 13:01:33.683638 12632 heartbeater.cc:511] Master 127.21.235.126:36127 requested a full tablet report, sending...
I20260809 13:01:33.685907 12425 ts_manager.cc:194] Registered new tserver with Master: 0c1f5f4b622641b29983ceb22d2f3016 (127.21.235.65:46609)
I20260809 13:01:33.687615 12425 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.235.65:36557
W20260809 13:01:34.017560 12636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:34.017863 12636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:34.017951 12636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:34.025771 12636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:34.025924 12636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:34.026036 12636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.66
I20260809 13:01:34.036303 12636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.235.66
--webserver_port=0
--tserver_master_addrs=127.21.235.126:36127
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.66
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:34.037894 12636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:34.039912 12636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:34.053367 12641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:34.053326 12642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:34.053293 12644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:34.059754 12636 server_base.cc:1034] running on GCE node
I20260809 13:01:34.060477 12636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:34.061542 12636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:34.062758 12636 hybrid_clock.cc:648] HybridClock initialized: now 1786280494062685 us; error 61 us; skew 500 ppm
I20260809 13:01:34.063110 12636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:01:34.065694 12636 webserver.cc:460] Webserver started at http://127.21.235.66:33039/ using document root <none> and password file <none>
I20260809 13:01:34.066591 12636 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:34.066689 12636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:34.067061 12636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:34.070488 12636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-1/data/instance:
uuid: "c6b72d17bf8e4016ace0706e3864a55e"
format_stamp: "Formatted at 2026-08-09 13:01:34 on dist-test-slave-gp6n"
server_key: "87fe327efd7e6c05c15390242cbadbd9"
server_key_iv: "4b2ec6c11cd31bdbb0c1f287e86b1666"
server_key_version: "encryptionkey@0"
I20260809 13:01:34.071511 12636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance:
uuid: "c6b72d17bf8e4016ace0706e3864a55e"
format_stamp: "Formatted at 2026-08-09 13:01:34 on dist-test-slave-gp6n"
server_key: "87fe327efd7e6c05c15390242cbadbd9"
server_key_iv: "4b2ec6c11cd31bdbb0c1f287e86b1666"
server_key_version: "encryptionkey@0"
I20260809 13:01:34.077993 12636 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260809 13:01:34.082199 12650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:34.083809 12636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:01:34.083962 12636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-1/wal
uuid: "c6b72d17bf8e4016ace0706e3864a55e"
format_stamp: "Formatted at 2026-08-09 13:01:34 on dist-test-slave-gp6n"
server_key: "87fe327efd7e6c05c15390242cbadbd9"
server_key_iv: "4b2ec6c11cd31bdbb0c1f287e86b1666"
server_key_version: "encryptionkey@0"
I20260809 13:01:34.084182 12636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:34.118700 12636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:34.119561 12636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:34.119889 12636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:34.120932 12636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:01:34.122488 12636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:01:34.122605 12636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:34.122740 12636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:01:34.122833 12636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:34.157166 12636 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.66:46425
I20260809 13:01:34.157274 12787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.66:46425 every 8 connection(s)
I20260809 13:01:34.158663 12636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-1/data/info.pb
I20260809 13:01:34.168071 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 12636
I20260809 13:01:34.168267 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-1/wal/instance
I20260809 13:01:34.168828 22445 external_mini_cluster.cc:1428] Setting key add41854d754462feb79ba0e0690f1f3
I20260809 13:01:34.171896 12634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:34.161289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.66:45505 (local address 127.21.235.126:36127)
0809 13:01:34.161506 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:34.161513 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:34.163000 (+ 1487us) server_negotiation.cc:407] Connection header received
0809 13:01:34.164773 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:34.164778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:34.164849 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:34.164947 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:34.165970 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.166755 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.168742 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.169170 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.169283 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:34.170938 (+ 1655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:34.170945 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:34.170950 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:34.170974 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:34.171016 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:34.171033 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:34.171133 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:34.171492 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 13:01:34.171604 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 13:01:34.172451 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.235.67:0
--local_ip_for_outbound_sockets=127.21.235.67
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:36127
--builtin_ntp_servers=127.21.235.84:33839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 13:01:34.173426 12791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:34.161851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:36127 (local address 127.21.235.66:45505)
0809 13:01:34.162819 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:34.162873 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:34.164520 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:34.165151 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:34.165158 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:34.165245 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:01:34.165812 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:34.165839 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.166904 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.166908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:34.168601 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:34.168615 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.169421 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.169430 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:34.169694 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:34.170572 (+ 878us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:01:34.170574 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:01:34.170615 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:01:34.170693 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:01:34.171185 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:01:34.171198 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:01:34.171207 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 13:01:34.171449 (+ 242us) client_negotiation.cc:240] Negotiation successful
0809 13:01:34.171911 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":277,"threads_started":1}
I20260809 13:01:34.179385 12789 heartbeater.cc:347] Connected to a master server at 127.21.235.126:36127
I20260809 13:01:34.179754 12789 heartbeater.cc:464] Registering TS with master...
I20260809 13:01:34.180794 12789 heartbeater.cc:511] Master 127.21.235.126:36127 requested a full tablet report, sending...
I20260809 13:01:34.182600 12425 ts_manager.cc:194] Registered new tserver with Master: c6b72d17bf8e4016ace0706e3864a55e (127.21.235.66:46425)
I20260809 13:01:34.183432 12425 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.235.66:45505
W20260809 13:01:34.542354 12792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:34.542707 12792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:34.542882 12792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:34.550520 12792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:34.550679 12792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:34.550869 12792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.235.67
I20260809 13:01:34.560930 12792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.235.67
--webserver_port=0
--tserver_master_addrs=127.21.235.126:36127
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.235.67
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:34.562242 12792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:34.564123 12792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:34.577553 12797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:34.577598 12798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:34.577564 12800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:34.578480 12792 server_base.cc:1034] running on GCE node
I20260809 13:01:34.579092 12792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:34.580142 12792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:34.581391 12792 hybrid_clock.cc:648] HybridClock initialized: now 1786280494581364 us; error 65 us; skew 500 ppm
I20260809 13:01:34.581709 12792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:01:34.584029 12792 webserver.cc:460] Webserver started at http://127.21.235.67:38171/ using document root <none> and password file <none>
I20260809 13:01:34.584726 12792 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:34.584859 12792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:34.585176 12792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:34.588169 12792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-2/data/instance:
uuid: "9af151a403f342baa238cae70a9dab3e"
format_stamp: "Formatted at 2026-08-09 13:01:34 on dist-test-slave-gp6n"
server_key: "05ad56dc1d54d264283f20a1019a53bd"
server_key_iv: "01fbcf68ad06c24c897cf2118f2d6406"
server_key_version: "encryptionkey@0"
I20260809 13:01:34.588977 12792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance:
uuid: "9af151a403f342baa238cae70a9dab3e"
format_stamp: "Formatted at 2026-08-09 13:01:34 on dist-test-slave-gp6n"
server_key: "05ad56dc1d54d264283f20a1019a53bd"
server_key_iv: "01fbcf68ad06c24c897cf2118f2d6406"
server_key_version: "encryptionkey@0"
I20260809 13:01:34.594048 12792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260809 13:01:34.597608 12807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:34.599032 12792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 13:01:34.599206 12792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-2/wal
uuid: "9af151a403f342baa238cae70a9dab3e"
format_stamp: "Formatted at 2026-08-09 13:01:34 on dist-test-slave-gp6n"
server_key: "05ad56dc1d54d264283f20a1019a53bd"
server_key_iv: "01fbcf68ad06c24c897cf2118f2d6406"
server_key_version: "encryptionkey@0"
I20260809 13:01:34.599448 12792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:34.622884 12792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:34.623746 12792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:34.624070 12792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:34.625087 12792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:01:34.626724 12792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:01:34.626844 12792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:34.626976 12792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:01:34.627063 12792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:34.660969 12792 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.67:42181
I20260809 13:01:34.661058 12937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.67:42181 every 8 connection(s)
I20260809 13:01:34.662341 12792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-2/data/info.pb
I20260809 13:01:34.667500 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 12792
I20260809 13:01:34.667702 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-2/wal/instance
I20260809 13:01:34.668161 22445 external_mini_cluster.cc:1428] Setting key 2f877cf6377ef84e02150a8b2bb07997
I20260809 13:01:34.675766 12634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:34.665270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.67:50797 (local address 127.21.235.126:36127)
0809 13:01:34.665496 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:34.665502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:34.667087 (+ 1585us) server_negotiation.cc:407] Connection header received
0809 13:01:34.669454 (+ 2367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:34.669460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:34.669556 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:34.669655 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:34.670994 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.671818 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.673341 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.673736 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.673841 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:34.675116 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:34.675120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:34.675124 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:34.675141 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:34.675172 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:34.675184 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:34.675258 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:34.675455 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 13:01:34.675531 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 13:01:34.676679 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:34.665807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:36127 (local address 127.21.235.67:50797)
0809 13:01:34.666887 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:34.666944 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:34.669087 (+ 2143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:34.669928 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:34.669936 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:34.670072 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:01:34.670774 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:34.670793 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.671966 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.671970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:34.673204 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:34.673215 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.673915 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.673921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:34.674112 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:34.674820 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:01:34.674823 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:01:34.674859 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:01:34.674940 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:01:34.675269 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:01:34.675277 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:01:34.675282 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 13:01:34.675407 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 13:01:34.675701 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":329,"threads_started":1}
I20260809 13:01:34.678006 12938 heartbeater.cc:347] Connected to a master server at 127.21.235.126:36127
I20260809 13:01:34.678295 12938 heartbeater.cc:464] Registering TS with master...
I20260809 13:01:34.679036 12938 heartbeater.cc:511] Master 127.21.235.126:36127 requested a full tablet report, sending...
I20260809 13:01:34.680377 12425 ts_manager.cc:194] Registered new tserver with Master: 9af151a403f342baa238cae70a9dab3e (127.21.235.67:42181)
I20260809 13:01:34.681102 12425 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.235.67:50797
I20260809 13:01:34.683615 22445 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:01:34.690958 12632 heartbeater.cc:503] Master 127.21.235.126:36127 was elected leader, sending a full tablet report...
I20260809 13:01:34.694556 12634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:34.688110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59628 (local address 127.21.235.126:36127)
0809 13:01:34.688378 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:34.688384 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:34.688653 (+ 269us) server_negotiation.cc:407] Connection header received
0809 13:01:34.688837 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:34.688841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:34.688902 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:34.688978 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:34.689454 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.690202 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.692011 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.692589 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.692734 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:34.693815 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:34.693820 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:34.693823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:34.693841 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:34.693872 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:34.693884 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:34.693976 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:34.694119 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 13:01:34.694213 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 13:01:34.698838 12425 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:01:34.706772 12425 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:01:34.732040 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:34.720151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:46425 (local address 127.0.0.1:59766)
0809 13:01:34.721057 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:34.721115 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:34.721363 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:34.723652 (+ 2289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:34.723661 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:34.723724 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:34.724434 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:34.724448 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.726532 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.726539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:34.729417 (+ 2878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:34.729434 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.731009 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.731015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:34.731198 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:34.731247 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 13:01:34.731405 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 13:01:34.731616 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":152,"spinlock_wait_cycles":3968,"thread_start_us":393,"threads_started":1}
I20260809 13:01:34.732064 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:34.720496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59766 (local address 127.21.235.66:46425)
0809 13:01:34.723037 (+ 2541us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:34.723045 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:34.723071 (+ 26us) server_negotiation.cc:407] Connection header received
0809 13:01:34.723166 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:34.723172 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:34.723356 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:34.723940 (+ 584us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:34.724678 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.726313 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.729622 (+ 3309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.730797 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.730972 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:34.731252 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:34.731464 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 13:01:34.731651 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2390,"thread_start_us":2234,"threads_started":1}
I20260809 13:01:34.732888 12956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:34.721952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41582 (local address 127.21.235.67:42181)
0809 13:01:34.722863 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:34.722882 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:34.722919 (+ 37us) server_negotiation.cc:407] Connection header received
0809 13:01:34.723026 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:34.723035 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:34.723322 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:34.724425 (+ 1103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:34.724726 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.726313 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.729375 (+ 3062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.730859 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.731018 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:34.731290 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:34.732332 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0809 13:01:34.732494 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":514,"threads_started":1}
I20260809 13:01:34.733883 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:34.721163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:46609 (local address 127.0.0.1:48066)
0809 13:01:34.723994 (+ 2831us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:34.724014 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:34.724233 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:34.725483 (+ 1250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:34.725489 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:34.725516 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:34.725952 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:34.725970 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.728207 (+ 2237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.728212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:34.730607 (+ 2395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:34.730622 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.732977 (+ 2355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.732987 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:34.733211 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:34.733246 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 13:01:34.733355 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 13:01:34.733507 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2684,"mutex_wait_us":204,"thread_start_us":2477,"threads_started":1}
I20260809 13:01:34.732205 12953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:34.720927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:42181 (local address 127.0.0.1:41582)
0809 13:01:34.722132 (+ 1205us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:34.722147 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:34.722238 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:34.723652 (+ 1414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:34.723661 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:34.723715 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:34.724478 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:34.724497 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.726576 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.726581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:34.729077 (+ 2496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:34.729092 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.731418 (+ 2326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.731424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:34.731616 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:34.731646 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 13:01:34.731728 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 13:01:34.731852 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1066,"mutex_wait_us":203,"spinlock_wait_cycles":768,"thread_start_us":920,"threads_started":1}
I20260809 13:01:34.734843 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:34.721622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48066 (local address 127.21.235.65:46609)
0809 13:01:34.723420 (+ 1798us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:34.723434 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:34.724118 (+ 684us) server_negotiation.cc:407] Connection header received
0809 13:01:34.724452 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:34.724462 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:34.724988 (+ 526us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:34.725203 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:34.726160 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.728045 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.730830 (+ 2785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.732419 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.732574 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:34.732870 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:34.734238 (+ 1368us) server_negotiation.cc:299] Negotiation successful
0809 13:01:34.734479 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1591,"thread_start_us":1450,"threads_started":1}
I20260809 13:01:34.736279 12708 tablet_service.cc:1467] Processing CreateTablet for tablet 98b3f6eaf1104ca59a9edbb2ec05db33 (DEFAULT_TABLE table=test-table [id=8f4e91c08276449aaedc0071002ccb6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:01:34.737498 12708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98b3f6eaf1104ca59a9edbb2ec05db33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:34.737454 12860 tablet_service.cc:1467] Processing CreateTablet for tablet 98b3f6eaf1104ca59a9edbb2ec05db33 (DEFAULT_TABLE table=test-table [id=8f4e91c08276449aaedc0071002ccb6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:01:34.738997 12860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98b3f6eaf1104ca59a9edbb2ec05db33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:34.740201 12559 tablet_service.cc:1467] Processing CreateTablet for tablet 98b3f6eaf1104ca59a9edbb2ec05db33 (DEFAULT_TABLE table=test-table [id=8f4e91c08276449aaedc0071002ccb6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:01:34.741525 12559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98b3f6eaf1104ca59a9edbb2ec05db33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:34.751500 12958 tablet_bootstrap.cc:492] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e: Bootstrap starting.
I20260809 13:01:34.755074 12958 tablet_bootstrap.cc:654] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:34.756649 12958 log.cc:824] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:34.758260 12959 tablet_bootstrap.cc:492] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e: Bootstrap starting.
I20260809 13:01:34.760365 12958 tablet_bootstrap.cc:492] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e: No bootstrap required, opened a new log
I20260809 13:01:34.760676 12958 ts_tablet_manager.cc:1397] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 13:01:34.763151 12960 tablet_bootstrap.cc:492] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016: Bootstrap starting.
I20260809 13:01:34.763566 12959 tablet_bootstrap.cc:654] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:34.764825 12959 log.cc:824] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:34.767143 12960 tablet_bootstrap.cc:654] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:34.768577 12959 tablet_bootstrap.cc:492] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e: No bootstrap required, opened a new log
I20260809 13:01:34.768529 12960 log.cc:824] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:34.768850 12959 ts_tablet_manager.cc:1397] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.002s
I20260809 13:01:34.771761 12960 tablet_bootstrap.cc:492] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016: No bootstrap required, opened a new log
I20260809 13:01:34.772096 12960 ts_tablet_manager.cc:1397] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 13:01:34.773118 12958 raft_consensus.cc:348] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b72d17bf8e4016ace0706e3864a55e" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46425 } } peers { permanent_uuid: "0c1f5f4b622641b29983ceb22d2f3016" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46609 } } peers { permanent_uuid: "9af151a403f342baa238cae70a9dab3e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42181 } }
I20260809 13:01:34.773619 12958 raft_consensus.cc:374] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:34.773736 12958 raft_consensus.cc:729] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9af151a403f342baa238cae70a9dab3e, State: Initialized, Role: FOLLOWER
I20260809 13:01:34.774435 12958 consensus_queue.cc:260] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b72d17bf8e4016ace0706e3864a55e" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46425 } } peers { permanent_uuid: "0c1f5f4b622641b29983ceb22d2f3016" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46609 } } peers { permanent_uuid: "9af151a403f342baa238cae70a9dab3e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42181 } }
I20260809 13:01:34.776473 12938 heartbeater.cc:503] Master 127.21.235.126:36127 was elected leader, sending a full tablet report...
I20260809 13:01:34.777817 12959 raft_consensus.cc:348] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b72d17bf8e4016ace0706e3864a55e" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46425 } } peers { permanent_uuid: "0c1f5f4b622641b29983ceb22d2f3016" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46609 } } peers { permanent_uuid: "9af151a403f342baa238cae70a9dab3e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42181 } }
I20260809 13:01:34.778318 12959 raft_consensus.cc:374] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:34.778496 12959 raft_consensus.cc:729] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6b72d17bf8e4016ace0706e3864a55e, State: Initialized, Role: FOLLOWER
I20260809 13:01:34.779090 12959 consensus_queue.cc:260] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b72d17bf8e4016ace0706e3864a55e" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46425 } } peers { permanent_uuid: "0c1f5f4b622641b29983ceb22d2f3016" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46609 } } peers { permanent_uuid: "9af151a403f342baa238cae70a9dab3e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42181 } }
I20260809 13:01:34.780232 12789 heartbeater.cc:503] Master 127.21.235.126:36127 was elected leader, sending a full tablet report...
I20260809 13:01:34.780922 12959 ts_tablet_manager.cc:1428] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260809 13:01:34.781477 12958 ts_tablet_manager.cc:1428] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e: Time spent starting tablet: real 0.021s user 0.009s sys 0.008s
I20260809 13:01:34.784286 12960 raft_consensus.cc:348] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b72d17bf8e4016ace0706e3864a55e" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46425 } } peers { permanent_uuid: "0c1f5f4b622641b29983ceb22d2f3016" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46609 } } peers { permanent_uuid: "9af151a403f342baa238cae70a9dab3e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42181 } }
I20260809 13:01:34.784762 12960 raft_consensus.cc:374] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:34.784869 12960 raft_consensus.cc:729] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c1f5f4b622641b29983ceb22d2f3016, State: Initialized, Role: FOLLOWER
I20260809 13:01:34.785462 12960 consensus_queue.cc:260] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b72d17bf8e4016ace0706e3864a55e" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46425 } } peers { permanent_uuid: "0c1f5f4b622641b29983ceb22d2f3016" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46609 } } peers { permanent_uuid: "9af151a403f342baa238cae70a9dab3e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42181 } }
I20260809 13:01:34.786849 12960 ts_tablet_manager.cc:1428] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260809 13:01:34.906286 12966 raft_consensus.cc:482] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:01:34.906688 12966 raft_consensus.cc:504] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b72d17bf8e4016ace0706e3864a55e" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46425 } } peers { permanent_uuid: "0c1f5f4b622641b29983ceb22d2f3016" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46609 } } peers { permanent_uuid: "9af151a403f342baa238cae70a9dab3e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42181 } }
I20260809 13:01:34.908586 12966 leader_election.cc:288] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6b72d17bf8e4016ace0706e3864a55e (127.21.235.66:46425), 9af151a403f342baa238cae70a9dab3e (127.21.235.67:42181)
W20260809 13:01:34.910770 12790 tablet.cc:2369] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:01:34.914109 12939 tablet.cc:2369] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:01:34.916157 12956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:34.909075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:36443 (local address 127.21.235.67:42181)
0809 13:01:34.909325 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:34.909334 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:34.910050 (+ 716us) server_negotiation.cc:407] Connection header received
0809 13:01:34.910250 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:34.910254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:34.910310 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:34.910406 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:34.911227 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.912249 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.914215 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.915237 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.915405 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:34.915476 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:34.915791 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 13:01:34.915913 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 13:01:34.916216 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:34.909071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.235.65:35261 (local address 127.21.235.66:46425)
0809 13:01:34.909297 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:34.909304 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:34.909853 (+ 549us) server_negotiation.cc:407] Connection header received
0809 13:01:34.910085 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:34.910089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:34.910163 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:34.910263 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:01:34.911131 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.912023 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.914177 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.915327 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.915525 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:34.915595 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:34.915855 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 13:01:34.915977 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 13:01:34.916268 12968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:34.909051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.67:42181 (local address 127.21.235.65:36443)
0809 13:01:34.909939 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:34.909954 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:34.910100 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:34.910459 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:34.910487 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:34.910528 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:34.911026 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:34.911039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.912393 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.912397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:34.913917 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:34.913930 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.915434 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.915440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:34.915627 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:34.915651 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 13:01:34.915754 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 13:01:34.915882 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":6,"thread_start_us":568,"threads_started":1}
I20260809 13:01:34.916329 12967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:34.908866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.66:46425 (local address 127.21.235.65:35261)
0809 13:01:34.909720 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:34.909741 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:01:34.909931 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:01:34.910283 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:01:34.910288 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:01:34.910326 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:01:34.910896 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:34.910911 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.912143 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.912149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:34.913908 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:01:34.913925 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.915518 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.915524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:01:34.915703 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:34.915725 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 13:01:34.915826 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 13:01:34.915946 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":85,"thread_start_us":482,"threads_started":1}
W20260809 13:01:34.916713 12633 tablet.cc:2369] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:01:34.917375 12735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98b3f6eaf1104ca59a9edbb2ec05db33" candidate_uuid: "0c1f5f4b622641b29983ceb22d2f3016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b72d17bf8e4016ace0706e3864a55e" is_pre_election: true
I20260809 13:01:34.917575 12887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98b3f6eaf1104ca59a9edbb2ec05db33" candidate_uuid: "0c1f5f4b622641b29983ceb22d2f3016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9af151a403f342baa238cae70a9dab3e" is_pre_election: true
I20260809 13:01:34.917992 12735 raft_consensus.cc:2454] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c1f5f4b622641b29983ceb22d2f3016 in term 0.
I20260809 13:01:34.918081 12887 raft_consensus.cc:2454] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c1f5f4b622641b29983ceb22d2f3016 in term 0.
I20260809 13:01:34.918766 12504 leader_election.cc:302] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c1f5f4b622641b29983ceb22d2f3016, 9af151a403f342baa238cae70a9dab3e; no voters:
I20260809 13:01:34.919183 12966 raft_consensus.cc:2781] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:01:34.919340 12966 raft_consensus.cc:482] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:01:34.919468 12966 raft_consensus.cc:3037] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:34.921559 12966 raft_consensus.cc:504] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b72d17bf8e4016ace0706e3864a55e" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46425 } } peers { permanent_uuid: "0c1f5f4b622641b29983ceb22d2f3016" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46609 } } peers { permanent_uuid: "9af151a403f342baa238cae70a9dab3e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42181 } }
I20260809 13:01:34.922118 12966 leader_election.cc:288] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [CANDIDATE]: Term 1 election: Requested vote from peers c6b72d17bf8e4016ace0706e3864a55e (127.21.235.66:46425), 9af151a403f342baa238cae70a9dab3e (127.21.235.67:42181)
I20260809 13:01:34.922564 12735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98b3f6eaf1104ca59a9edbb2ec05db33" candidate_uuid: "0c1f5f4b622641b29983ceb22d2f3016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b72d17bf8e4016ace0706e3864a55e"
I20260809 13:01:34.922590 12887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98b3f6eaf1104ca59a9edbb2ec05db33" candidate_uuid: "0c1f5f4b622641b29983ceb22d2f3016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9af151a403f342baa238cae70a9dab3e"
I20260809 13:01:34.922757 12735 raft_consensus.cc:3037] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:34.922793 12887 raft_consensus.cc:3037] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:34.925019 12887 raft_consensus.cc:2454] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c1f5f4b622641b29983ceb22d2f3016 in term 1.
I20260809 13:01:34.925026 12735 raft_consensus.cc:2454] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c1f5f4b622641b29983ceb22d2f3016 in term 1.
I20260809 13:01:34.925468 12504 leader_election.cc:302] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c1f5f4b622641b29983ceb22d2f3016, 9af151a403f342baa238cae70a9dab3e; no voters:
I20260809 13:01:34.925777 12966 raft_consensus.cc:2781] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:34.926291 12966 raft_consensus.cc:686] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [term 1 LEADER]: Becoming Leader. State: Replica: 0c1f5f4b622641b29983ceb22d2f3016, State: Running, Role: LEADER
I20260809 13:01:34.926807 12966 consensus_queue.cc:237] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b72d17bf8e4016ace0706e3864a55e" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46425 } } peers { permanent_uuid: "0c1f5f4b622641b29983ceb22d2f3016" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46609 } } peers { permanent_uuid: "9af151a403f342baa238cae70a9dab3e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42181 } }
I20260809 13:01:34.931804 12425 catalog_manager.cc:5495] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c1f5f4b622641b29983ceb22d2f3016 (127.21.235.65). New cstate: current_term: 1 leader_uuid: "0c1f5f4b622641b29983ceb22d2f3016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b72d17bf8e4016ace0706e3864a55e" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c1f5f4b622641b29983ceb22d2f3016" member_type: VOTER last_known_addr { host: "127.21.235.65" port: 46609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9af151a403f342baa238cae70a9dab3e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42181 } health_report { overall_health: UNKNOWN } } }
I20260809 13:01:34.977829 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:34.971674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48072 (local address 127.21.235.65:46609)
0809 13:01:34.971909 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:34.971917 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:34.972014 (+ 97us) server_negotiation.cc:407] Connection header received
0809 13:01:34.972305 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:34.972310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:34.972445 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:34.972619 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:01:34.973038 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.973870 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.975497 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:34.975972 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:34.976122 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:34.976644 (+ 522us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:01:34.977199 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:01:34.977395 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:34.977462 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 13:01:34.977550 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 13:01:34.986610 12887 raft_consensus.cc:1261] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e [term 1 FOLLOWER]: Refusing update from remote peer 0c1f5f4b622641b29983ceb22d2f3016: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:01:34.986683 12735 raft_consensus.cc:1261] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e [term 1 FOLLOWER]: Refusing update from remote peer 0c1f5f4b622641b29983ceb22d2f3016: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:01:34.987578 12969 consensus_queue.cc:1035] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9af151a403f342baa238cae70a9dab3e" member_type: VOTER last_known_addr { host: "127.21.235.67" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:01:34.987960 12966 consensus_queue.cc:1035] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b72d17bf8e4016ace0706e3864a55e" member_type: VOTER last_known_addr { host: "127.21.235.66" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:01:35.000439 12974 mvcc.cc:205] Tried to move back new op lower bound from 7316604907454201856 to 7316604907224145920. Current Snapshot: MvccSnapshot[applied={T|T < 7316604907454201856}]
I20260809 13:01:35.017015 12424 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:01:35.017395 12424 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:01:35.021644 12424 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c5a28ce35fda4db69fd4922dace2eb39: Sending DeleteTablet for 3 replicas of tablet 98b3f6eaf1104ca59a9edbb2ec05db33
I20260809 13:01:35.022886 12559 tablet_service.cc:1514] Processing DeleteTablet for tablet 98b3f6eaf1104ca59a9edbb2ec05db33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:01:35 UTC) from {username='slave'} at 127.0.0.1:48066
I20260809 13:01:35.022933 12708 tablet_service.cc:1514] Processing DeleteTablet for tablet 98b3f6eaf1104ca59a9edbb2ec05db33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:01:35 UTC) from {username='slave'} at 127.0.0.1:59766
I20260809 13:01:35.023069 12860 tablet_service.cc:1514] Processing DeleteTablet for tablet 98b3f6eaf1104ca59a9edbb2ec05db33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:01:35 UTC) from {username='slave'} at 127.0.0.1:41582
I20260809 13:01:35.024901 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 12489
I20260809 13:01:35.038525 12984 tablet_replica.cc:330] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e: stopping tablet replica
I20260809 13:01:35.039283 12984 raft_consensus.cc:2229] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:01:35.040036 12984 raft_consensus.cc:2258] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:01:35.042779 12983 tablet_replica.cc:330] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016: stopping tablet replica
I20260809 13:01:35.042747 12984 ts_tablet_manager.cc:1905] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:01:35.043370 12983 raft_consensus.cc:2229] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:01:35.046162 12983 raft_consensus.cc:2258] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:01:35.049446 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:35.026659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48078 (local address 127.21.235.65:46609)
0809 13:01:35.026860 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:35.026871 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:35.037011 (+ 10140us) server_negotiation.cc:407] Connection header received
0809 13:01:35.037672 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:35.037682 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:35.037829 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:35.037930 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:35.038841 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:35.040144 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:35.043688 (+ 3544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:35.044426 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:35.044544 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:35.047129 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:35.047141 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:35.047159 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:35.048053 (+ 894us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:35.048120 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:35.048147 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:35.048257 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:35.048869 (+ 612us) server_negotiation.cc:299] Negotiation successful
0809 13:01:35.048989 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 13:01:35.050911 12984 ts_tablet_manager.cc:1918] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:01:35.051116 12627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:35.051172 12984 log.cc:1196] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-1/wal/wals/98b3f6eaf1104ca59a9edbb2ec05db33
I20260809 13:01:35.051842 12984 ts_tablet_manager.cc:1939] T 98b3f6eaf1104ca59a9edbb2ec05db33 P c6b72d17bf8e4016ace0706e3864a55e: Deleting consensus metadata
I20260809 13:01:35.052175 12985 tablet_replica.cc:330] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e: stopping tablet replica
I20260809 13:01:35.052767 12985 raft_consensus.cc:2229] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:01:35.053433 12985 raft_consensus.cc:2258] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:01:35.054783 12410 catalog_manager.cc:4841] TS c6b72d17bf8e4016ace0706e3864a55e (127.21.235.66:46425): tablet 98b3f6eaf1104ca59a9edbb2ec05db33 (table test-table [id=8f4e91c08276449aaedc0071002ccb6f]) successfully deleted
I20260809 13:01:35.056056 12985 ts_tablet_manager.cc:1905] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:01:35.061280 12985 ts_tablet_manager.cc:1918] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:01:35.061455 12985 log.cc:1196] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e: Deleting WAL directory at /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280163688078-22445-0/minicluster-data/ts-2/wal/wals/98b3f6eaf1104ca59a9edbb2ec05db33
I20260809 13:01:35.061904 12985 ts_tablet_manager.cc:1939] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 9af151a403f342baa238cae70a9dab3e: Deleting consensus metadata
I20260809 13:01:35.063529 12408 catalog_manager.cc:4841] TS 9af151a403f342baa238cae70a9dab3e (127.21.235.67:42181): tablet 98b3f6eaf1104ca59a9edbb2ec05db33 (table test-table [id=8f4e91c08276449aaedc0071002ccb6f]) successfully deleted
I20260809 13:01:35.174715 12983 ts_tablet_manager.cc:1905] T 98b3f6eaf1104ca59a9edbb2ec05db33 P 0c1f5f4b622641b29983ceb22d2f3016: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:01:35.178426 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 12489
W20260809 13:01:35.203202 12409 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.235.65:46609 (error 108)
I20260809 13:01:35.204187 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 12636
I20260809 13:01:35.205385 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:35.204643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:46609 (local address 127.0.0.1:48088)
0809 13:01:35.204942 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:35.205134 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:46609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260809 13:01:35.205698 12409 catalog_manager.cc:4584] TS 0c1f5f4b622641b29983ceb22d2f3016 (127.21.235.65:46609): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 98b3f6eaf1104ca59a9edbb2ec05db33: Network error: Client connection negotiation failed: client connection to 127.21.235.65:46609: connect: Connection refused (error 111)
I20260809 13:01:35.211843 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:35.205100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59778 (local address 127.21.235.66:46425)
0809 13:01:35.205274 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:35.205281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:35.205343 (+ 62us) server_negotiation.cc:407] Connection header received
0809 13:01:35.205640 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:35.205646 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:35.205756 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:35.205871 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:35.206349 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:35.207182 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:35.208447 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:35.208890 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:35.209028 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:35.210195 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:35.210208 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:35.210227 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:35.210967 (+ 740us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:35.211063 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:35.211094 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:35.211211 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:35.211350 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 13:01:35.211456 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 13:01:35.212568 12782 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:35.259738 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:35.259163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:46609 (local address 127.0.0.1:48090)
0809 13:01:35.259432 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:35.259544 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:46609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 13:01:35.261059 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:35.260542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:46609 (local address 127.0.0.1:48094)
0809 13:01:35.260736 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:35.260859 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:46609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 13:01:35.304163 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 12636
I20260809 13:01:35.321561 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:35.321049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:46609 (local address 127.0.0.1:48110)
0809 13:01:35.321268 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:35.321354 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:46609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 13:01:35.322686 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:35.322304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:46609 (local address 127.0.0.1:48124)
0809 13:01:35.322482 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:35.322563 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:46609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 13:01:35.335319 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 12792
I20260809 13:01:35.343909 12956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:35.336201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41590 (local address 127.21.235.67:42181)
0809 13:01:35.336376 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:35.336381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:35.336499 (+ 118us) server_negotiation.cc:407] Connection header received
0809 13:01:35.336721 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:35.336725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:35.336827 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:35.336915 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:35.337487 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:35.338181 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:35.340477 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:35.340874 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:35.340976 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:35.342338 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:35.342347 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:35.342362 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:35.343104 (+ 742us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:35.343157 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:35.343178 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:35.343265 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:35.343468 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 13:01:35.343559 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 13:01:35.344547 12933 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:35.403541 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:35.402956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:46609 (local address 127.0.0.1:48132)
0809 13:01:35.403182 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:35.403346 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:46609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 13:01:35.404786 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:35.404335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.65:46609 (local address 127.0.0.1:48146)
0809 13:01:35.404560 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 13:01:35.404649 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.65:46609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260809 13:01:35.439940 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 12792
I20260809 13:01:35.459514 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 12392
I20260809 13:01:35.460798 12459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:35.571120 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 12392
2026-08-09T13:01:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3060 ms)
[----------] 6 tests from EncryptionPolicyTest (22557 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:01:35 dist-test-slave-gp6n krb5kdc[12995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:01:35 dist-test-slave-gp6n krb5kdc[12995](info): set up 2 sockets
Aug 09 13:01:35 dist-test-slave-gp6n krb5kdc[12995](info): commencing operation
krb5kdc: starting...
W20260809 13:01:36.777072 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:01:36 dist-test-slave-gp6n krb5kdc[12995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:01:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:01:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:01:37.095207 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33901
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:37.439944 13013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:37.440277 13013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:37.440447 13013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:37.447618 13013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:01:37.447773 13013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:37.447873 13013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:37.447986 13013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:01:37.448065 13013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:01:37.458726 13013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33901
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:37.460091 13013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:37.461494 13013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:37.473735 13021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:37.473708 13022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:37.473786 13024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:37.474892 13013 server_base.cc:1034] running on GCE node
I20260809 13:01:37.475705 13013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:37.477108 13013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:37.478338 13013 hybrid_clock.cc:648] HybridClock initialized: now 1786280497478288 us; error 44 us; skew 500 ppm
I20260809 13:01:37.478698 13013 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:37 dist-test-slave-gp6n krb5kdc[12995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280497, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:37.481842 13013 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:01:37.483429 13013 webserver.cc:460] Webserver started at http://127.0.0.1:37171/ using document root <none> and password file <none>
I20260809 13:01:37.484097 13013 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:37.484244 13013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:37.484556 13013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:37.487550 13013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "c94e700c55934b52b7e605ec54ed7d30"
format_stamp: "Formatted at 2026-08-09 13:01:37 on dist-test-slave-gp6n"
server_key: "e62da0cdaf0f32eec120b8b0e0c26fa4"
server_key_iv: "e1662ab48a367de70f8b299af4f70a84"
server_key_version: "encryptionkey@0"
I20260809 13:01:37.488332 13013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "c94e700c55934b52b7e605ec54ed7d30"
format_stamp: "Formatted at 2026-08-09 13:01:37 on dist-test-slave-gp6n"
server_key: "e62da0cdaf0f32eec120b8b0e0c26fa4"
server_key_iv: "e1662ab48a367de70f8b299af4f70a84"
server_key_version: "encryptionkey@0"
I20260809 13:01:37.493553 13013 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:01:37.497112 13031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:37.498654 13013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:01:37.498854 13013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "c94e700c55934b52b7e605ec54ed7d30"
format_stamp: "Formatted at 2026-08-09 13:01:37 on dist-test-slave-gp6n"
server_key: "e62da0cdaf0f32eec120b8b0e0c26fa4"
server_key_iv: "e1662ab48a367de70f8b299af4f70a84"
server_key_version: "encryptionkey@0"
I20260809 13:01:37.499069 13013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:37.516956 13013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:37.521838 13013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:37.522174 13013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:37.540798 13013 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33901
I20260809 13:01:37.540822 13091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33901 every 8 connection(s)
I20260809 13:01:37.542274 13013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:01:37.544809 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 13013
I20260809 13:01:37.545042 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:01:37.545854 22445 external_mini_cluster.cc:1428] Setting key cc078ae7852518c4eb0a929acae8458e
I20260809 13:01:37.547061 13092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 13:01:37 dist-test-slave-gp6n krb5kdc[12995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 13:01:37.562500 13092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30: Bootstrap starting.
I20260809 13:01:37.565726 13092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:37.566895 13092 log.cc:824] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:37.570523 13092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30: No bootstrap required, opened a new log
I20260809 13:01:37.571466 13095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:37.548000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35867 (local address 127.0.0.1:33901)
0809 13:01:37.548870 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:37.548883 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:37.548926 (+ 43us) server_negotiation.cc:407] Connection header received
0809 13:01:37.550768 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:37.550792 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:37.551248 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:37.551618 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:37.553568 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:37.555087 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:37.556937 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:37.557626 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:37.557779 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:37.563076 (+ 5297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:37.563086 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:37.563100 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:37.563137 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:37.566303 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:37.567186 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:37.567194 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:37.567201 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:37.567437 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:37.568079 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:37.568083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:37.568085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:37.568851 (+ 766us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:37.569146 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:37.569636 (+ 490us) server_negotiation.cc:299] Negotiation successful
0809 13:01:37.569984 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":416,"threads_started":1}
I20260809 13:01:37.575613 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:01:37.580605 13092 raft_consensus.cc:348] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94e700c55934b52b7e605ec54ed7d30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33901 } }
I20260809 13:01:37.581090 13092 raft_consensus.cc:374] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:37.581200 13092 raft_consensus.cc:729] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c94e700c55934b52b7e605ec54ed7d30, State: Initialized, Role: FOLLOWER
I20260809 13:01:37.581881 13092 consensus_queue.cc:260] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94e700c55934b52b7e605ec54ed7d30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33901 } }
I20260809 13:01:37.582185 13092 raft_consensus.cc:388] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:01:37.582345 13092 raft_consensus.cc:482] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:01:37.582594 13092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:37.585783 13092 raft_consensus.cc:504] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94e700c55934b52b7e605ec54ed7d30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33901 } }
I20260809 13:01:37.586369 13092 leader_election.cc:302] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c94e700c55934b52b7e605ec54ed7d30; no voters:
I20260809 13:01:37.587064 13092 leader_election.cc:288] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:01:37.587174 13105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:37.587853 13105 raft_consensus.cc:686] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30 [term 1 LEADER]: Becoming Leader. State: Replica: c94e700c55934b52b7e605ec54ed7d30, State: Running, Role: LEADER
I20260809 13:01:37.588493 13105 consensus_queue.cc:237] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94e700c55934b52b7e605ec54ed7d30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33901 } }
I20260809 13:01:37.590747 13105 sys_catalog.cc:455] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c94e700c55934b52b7e605ec54ed7d30. Latest consensus state: current_term: 1 leader_uuid: "c94e700c55934b52b7e605ec54ed7d30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94e700c55934b52b7e605ec54ed7d30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33901 } } }
I20260809 13:01:37.591260 13105 sys_catalog.cc:458] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:37.591545 13107 sys_catalog.cc:455] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c94e700c55934b52b7e605ec54ed7d30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94e700c55934b52b7e605ec54ed7d30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33901 } } }
I20260809 13:01:37.591904 13107 sys_catalog.cc:458] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:37.592425 13092 sys_catalog.cc:564] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:01:37.598791 13110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:01:37.601954 13095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:37.581992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40639 (local address 127.0.0.1:33901)
0809 13:01:37.582203 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:37.582211 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:37.582800 (+ 589us) server_negotiation.cc:407] Connection header received
0809 13:01:37.583004 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:37.583008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:37.583073 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:37.583160 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:37.585027 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:37.585868 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:37.587940 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:37.588960 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:37.589101 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:37.593096 (+ 3995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:37.593105 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:37.593113 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:37.593169 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:37.597787 (+ 4618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:37.598965 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:37.598970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:37.598973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:37.599153 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:37.599880 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:37.599885 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:37.599888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:37.600313 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:37.600648 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:37.601347 (+ 699us) server_negotiation.cc:299] Negotiation successful
0809 13:01:37.601549 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 13:01:37.606979 13110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:01:37.616118 13110 catalog_manager.cc:1347] Generated new cluster ID: aec8b6cd06764b90bf8eb45f6f8e083a
I20260809 13:01:37.616245 13110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:01:37.637511 13110 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:01:37.638900 13110 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:01:37.646220 13110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c94e700c55934b52b7e605ec54ed7d30: Generated new TSK 0
I20260809 13:01:37.647100 13110 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:01:37.662034 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 13013
I20260809 13:01:37.663679 13086 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:37.756072 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 13013
W20260809 13:01:37.776494 12993 connection.cc:535] client connection to 127.0.0.1:33901 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33901): Connection reset by peer (error 104)
2026-08-09T13:01:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2177 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:01:37 dist-test-slave-gp6n krb5kdc[13129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:01:37 dist-test-slave-gp6n krb5kdc[13129](info): set up 2 sockets
Aug 09 13:01:37 dist-test-slave-gp6n krb5kdc[13129](info): commencing operation
krb5kdc: starting...
W20260809 13:01:38.952896 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:01:39 dist-test-slave-gp6n krb5kdc[13129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:01:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:01:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:01:39.269635 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39103
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:39.618193 13142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:39.618592 13142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:39.618755 13142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:39.625974 13142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:01:39.626127 13142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:39.626231 13142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:39.626344 13142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:01:39.626449 13142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:01:39.636361 13142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39103
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:39.637678 13142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:39.639091 13142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:39.651301 13148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:39.651228 13151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:39.651338 13149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:39.652817 13142 server_base.cc:1034] running on GCE node
I20260809 13:01:39.653579 13142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:39.655067 13142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:39.656316 13142 hybrid_clock.cc:648] HybridClock initialized: now 1786280499656260 us; error 43 us; skew 500 ppm
I20260809 13:01:39.656677 13142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:39 dist-test-slave-gp6n krb5kdc[13129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280499, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:39.660099 13142 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:01:39.661563 13142 webserver.cc:460] Webserver started at http://127.0.0.1:40379/ using document root <none> and password file <none>
I20260809 13:01:39.662247 13142 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:39.662446 13142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:39.662846 13142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:39.665880 13142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "b99b4a36151c4565933e8a0f6a029e57"
format_stamp: "Formatted at 2026-08-09 13:01:39 on dist-test-slave-gp6n"
server_key: "004fa5c0c757888bceab8e89f7d3092b"
server_key_iv: "3d24d3de140afa2c3116dacf81841695"
server_key_version: "encryptionkey@0"
I20260809 13:01:39.666755 13142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "b99b4a36151c4565933e8a0f6a029e57"
format_stamp: "Formatted at 2026-08-09 13:01:39 on dist-test-slave-gp6n"
server_key: "004fa5c0c757888bceab8e89f7d3092b"
server_key_iv: "3d24d3de140afa2c3116dacf81841695"
server_key_version: "encryptionkey@0"
I20260809 13:01:39.672228 13142 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 13:01:39.675804 13158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:39.677194 13142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:01:39.677384 13142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "b99b4a36151c4565933e8a0f6a029e57"
format_stamp: "Formatted at 2026-08-09 13:01:39 on dist-test-slave-gp6n"
server_key: "004fa5c0c757888bceab8e89f7d3092b"
server_key_iv: "3d24d3de140afa2c3116dacf81841695"
server_key_version: "encryptionkey@0"
I20260809 13:01:39.677587 13142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:39.688563 13142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:39.692762 13142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:39.693059 13142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:39.712550 13142 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39103
I20260809 13:01:39.712635 13216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39103 every 8 connection(s)
I20260809 13:01:39.714004 13142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:01:39.718010 13217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:39.719738 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 13142
I20260809 13:01:39.719985 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:01:39.720477 22445 external_mini_cluster.cc:1428] Setting key 2a658feaed7da2a1e481a4a3ddf92301
I20260809 13:01:39.732362 13217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57: Bootstrap starting.
I20260809 13:01:39.734279 13220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:39.722208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39953 (local address 127.0.0.1:39103)
0809 13:01:39.723137 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:39.723148 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:39.723195 (+ 47us) server_negotiation.cc:407] Connection header received
0809 13:01:39.724759 (+ 1564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:39.724785 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:39.725188 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:39.725516 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:39.726090 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:39.727401 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:39.729388 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:39.730096 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:39.730224 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:39.731619 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:39.731627 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:39.731640 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:39.732305 (+ 665us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:39.732453 (+ 148us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:39.732488 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:39.732656 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:39.733107 (+ 451us) server_negotiation.cc:299] Negotiation successful
0809 13:01:39.733299 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":437,"threads_started":1}
I20260809 13:01:39.735780 13217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:39.736917 13217 log.cc:824] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:39.738107 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:01:39.740185 13217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57: No bootstrap required, opened a new log
I20260809 13:01:39.749456 13220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:39.742711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37227 (local address 127.0.0.1:39103)
0809 13:01:39.742910 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:39.742917 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:39.743311 (+ 394us) server_negotiation.cc:407] Connection header received
0809 13:01:39.743457 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:39.743463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:39.743541 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:39.743649 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:39.744065 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:39.745213 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:39.746669 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:39.747417 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:39.747532 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:39.748607 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:39.748618 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:39.748623 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:39.748652 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:39.748713 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:39.748738 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:39.748849 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:39.749050 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 13:01:39.749110 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 13:01:39.750164 13217 raft_consensus.cc:348] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b99b4a36151c4565933e8a0f6a029e57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39103 } }
I20260809 13:01:39.750641 13217 raft_consensus.cc:374] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:39.750764 13217 raft_consensus.cc:729] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b99b4a36151c4565933e8a0f6a029e57, State: Initialized, Role: FOLLOWER
I20260809 13:01:39.751431 13217 consensus_queue.cc:260] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b99b4a36151c4565933e8a0f6a029e57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39103 } }
I20260809 13:01:39.751686 13217 raft_consensus.cc:388] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:01:39.751843 13217 raft_consensus.cc:482] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:01:39.752035 13217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:39.754278 13217 raft_consensus.cc:504] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b99b4a36151c4565933e8a0f6a029e57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39103 } }
I20260809 13:01:39.754828 13217 leader_election.cc:302] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b99b4a36151c4565933e8a0f6a029e57; no voters:
I20260809 13:01:39.755442 13217 leader_election.cc:288] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:01:39.755548 13228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:39.757041 13228 raft_consensus.cc:686] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57 [term 1 LEADER]: Becoming Leader. State: Replica: b99b4a36151c4565933e8a0f6a029e57, State: Running, Role: LEADER
I20260809 13:01:39.757563 13228 consensus_queue.cc:237] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b99b4a36151c4565933e8a0f6a029e57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39103 } }
I20260809 13:01:39.757766 13217 sys_catalog.cc:564] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:01:39.760577 13229 sys_catalog.cc:455] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b99b4a36151c4565933e8a0f6a029e57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b99b4a36151c4565933e8a0f6a029e57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39103 } } }
I20260809 13:01:39.760682 13230 sys_catalog.cc:455] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b99b4a36151c4565933e8a0f6a029e57. Latest consensus state: current_term: 1 leader_uuid: "b99b4a36151c4565933e8a0f6a029e57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b99b4a36151c4565933e8a0f6a029e57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39103 } } }
I20260809 13:01:39.760993 13229 sys_catalog.cc:458] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:39.761060 13230 sys_catalog.cc:458] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:39.761955 13237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:01:39.768246 13237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:01:39.777650 13237 catalog_manager.cc:1347] Generated new cluster ID: c76655d9829a4a949138c6abb7895d79
I20260809 13:01:39.777788 13237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:01:39.804697 13237 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:01:39.806648 13237 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:01:39.815958 13237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b99b4a36151c4565933e8a0f6a029e57: Generated new TSK 0
I20260809 13:01:39.817000 13237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:01:39.824800 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 13142
W20260809 13:01:39.825958 13212 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39953
W20260809 13:01:39.826762 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 13142: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:01:39.826973 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 13142
W20260809 13:01:39.849013 13127 connection.cc:535] client connection to 127.0.0.1:39103 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39103): Connection reset by peer (error 104)
2026-08-09T13:01:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2072 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:01:39 dist-test-slave-gp6n krb5kdc[13251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:01:39 dist-test-slave-gp6n krb5kdc[13251](info): set up 2 sockets
Aug 09 13:01:39 dist-test-slave-gp6n krb5kdc[13251](info): commencing operation
krb5kdc: starting...
W20260809 13:01:41.016635 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:01:41 dist-test-slave-gp6n krb5kdc[13251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:01:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:01:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:01:41.334040 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37675
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:41.680248 13268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:41.680614 13268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:41.680794 13268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:41.688109 13268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:01:41.688261 13268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:41.688377 13268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:41.688498 13268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:01:41.688578 13268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:01:41.698608 13268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37675
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:41.699913 13268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:41.701217 13268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:41.713579 13274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:41.713598 13276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:41.713598 13273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:41.714830 13268 server_base.cc:1034] running on GCE node
I20260809 13:01:41.715648 13268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:41.717049 13268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:41.718253 13268 hybrid_clock.cc:648] HybridClock initialized: now 1786280501718218 us; error 27 us; skew 500 ppm
I20260809 13:01:41.718652 13268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:41 dist-test-slave-gp6n krb5kdc[13251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280501, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:41.721962 13268 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:01:41.723498 13268 webserver.cc:460] Webserver started at http://127.0.0.1:35593/ using document root <none> and password file <none>
I20260809 13:01:41.724223 13268 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:41.724326 13268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:41.724581 13268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:41.727566 13268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "3198f3a363004ca4aef34b3817076f20"
format_stamp: "Formatted at 2026-08-09 13:01:41 on dist-test-slave-gp6n"
server_key: "d9735b6b10280f8694afa8cfbbee811f"
server_key_iv: "e86d01af8a345fa95bedb4f3fa86c66f"
server_key_version: "encryptionkey@0"
I20260809 13:01:41.728293 13268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "3198f3a363004ca4aef34b3817076f20"
format_stamp: "Formatted at 2026-08-09 13:01:41 on dist-test-slave-gp6n"
server_key: "d9735b6b10280f8694afa8cfbbee811f"
server_key_iv: "e86d01af8a345fa95bedb4f3fa86c66f"
server_key_version: "encryptionkey@0"
I20260809 13:01:41.733319 13268 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 13:01:41.737237 13284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:41.738586 13268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:01:41.738766 13268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "3198f3a363004ca4aef34b3817076f20"
format_stamp: "Formatted at 2026-08-09 13:01:41 on dist-test-slave-gp6n"
server_key: "d9735b6b10280f8694afa8cfbbee811f"
server_key_iv: "e86d01af8a345fa95bedb4f3fa86c66f"
server_key_version: "encryptionkey@0"
I20260809 13:01:41.738969 13268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:41.744786 13268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:41.748850 13268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:41.749159 13268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:41.767865 13268 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37675
I20260809 13:01:41.767912 13341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37675 every 8 connection(s)
I20260809 13:01:41.769307 13268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:01:41.773169 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:41.774843 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 13268
I20260809 13:01:41.775035 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:01:41.775460 22445 external_mini_cluster.cc:1428] Setting key f35971413a0225acbe8582e591c4ab35
I20260809 13:01:41.783177 13347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:41.777263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34073 (local address 127.0.0.1:37675)
0809 13:01:41.778153 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:41.778164 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:41.778204 (+ 40us) server_negotiation.cc:407] Connection header received
0809 13:01:41.779841 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:41.779865 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:41.780208 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:41.780518 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:41.781588 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:41.781595 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:41.781607 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:41.781621 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:41.781680 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:41.781698 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:41.781867 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:41.782003 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 13:01:41.782197 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":408,"threads_started":1}
I20260809 13:01:41.786284 13342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20: Bootstrap starting.
I20260809 13:01:41.786839 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:01:41.790046 13342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:41.791538 13342 log.cc:824] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:41.794778 13347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:41.792175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33765 (local address 127.0.0.1:37675)
0809 13:01:41.792398 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:41.792407 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:41.792851 (+ 444us) server_negotiation.cc:407] Connection header received
0809 13:01:41.793040 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:41.793045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:41.793106 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:41.793210 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:41.794122 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:41.794130 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:41.794137 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:41.794160 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:41.794205 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:41.794224 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:41.794334 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:41.794446 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 13:01:41.794502 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 13:01:41.795008 13342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20: No bootstrap required, opened a new log
I20260809 13:01:41.802946 13342 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3198f3a363004ca4aef34b3817076f20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37675 } }
I20260809 13:01:41.803315 13342 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:41.803431 13342 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3198f3a363004ca4aef34b3817076f20, State: Initialized, Role: FOLLOWER
I20260809 13:01:41.803977 13342 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3198f3a363004ca4aef34b3817076f20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37675 } }
I20260809 13:01:41.804206 13342 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:01:41.804355 13342 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:01:41.804525 13342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:41.806948 13342 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3198f3a363004ca4aef34b3817076f20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37675 } }
I20260809 13:01:41.807521 13342 leader_election.cc:302] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3198f3a363004ca4aef34b3817076f20; no voters:
I20260809 13:01:41.808063 13342 leader_election.cc:288] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:01:41.808207 13355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:41.809300 13355 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20 [term 1 LEADER]: Becoming Leader. State: Replica: 3198f3a363004ca4aef34b3817076f20, State: Running, Role: LEADER
I20260809 13:01:41.809801 13355 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3198f3a363004ca4aef34b3817076f20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37675 } }
I20260809 13:01:41.809999 13342 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:01:41.812659 13356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3198f3a363004ca4aef34b3817076f20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3198f3a363004ca4aef34b3817076f20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37675 } } }
I20260809 13:01:41.813112 13356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:41.813513 13357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3198f3a363004ca4aef34b3817076f20. Latest consensus state: current_term: 1 leader_uuid: "3198f3a363004ca4aef34b3817076f20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3198f3a363004ca4aef34b3817076f20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37675 } } }
I20260809 13:01:41.813784 13357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:41.814879 13364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:01:41.820595 13364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:01:41.832360 13364 catalog_manager.cc:1347] Generated new cluster ID: 39ffe114bb7148f8a5e996e60d797de8
I20260809 13:01:41.832505 13364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:01:41.847231 13364 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:01:41.847718 13364 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:01:41.860879 13364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3198f3a363004ca4aef34b3817076f20: Generated new TSK 0
I20260809 13:01:41.861989 13364 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:01:41.867344 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 13268
W20260809 13:01:41.868726 13337 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34073
W20260809 13:01:41.869498 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 13268: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:01:41.869657 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 13268
2026-08-09T13:01:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2041 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:01:41 dist-test-slave-gp6n krb5kdc[13377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:01:41 dist-test-slave-gp6n krb5kdc[13377](info): set up 2 sockets
Aug 09 13:01:41 dist-test-slave-gp6n krb5kdc[13377](info): commencing operation
krb5kdc: starting...
W20260809 13:01:43.104378 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:01:43 dist-test-slave-gp6n krb5kdc[13377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:01:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:01:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:01:43.429519 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45809
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:43.776465 13398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:43.776842 13398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:43.777010 13398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:43.784358 13398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:01:43.784513 13398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:43.784632 13398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:43.784746 13398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:01:43.784826 13398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:01:43.794723 13398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:43.796105 13398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:43.797523 13398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:43.811633 13403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:43.811903 13404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:43.812345 13406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:43.812834 13398 server_base.cc:1034] running on GCE node
I20260809 13:01:43.813642 13398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:43.815207 13398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:43.816468 13398 hybrid_clock.cc:648] HybridClock initialized: now 1786280503816420 us; error 59 us; skew 500 ppm
I20260809 13:01:43.816854 13398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:43 dist-test-slave-gp6n krb5kdc[13377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280503, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:43.825450 13398 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:01:43.827090 13398 webserver.cc:460] Webserver started at http://127.0.0.1:46011/ using document root <none> and password file <none>
I20260809 13:01:43.827770 13398 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:43.827878 13398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:43.828140 13398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:43.831310 13398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "b9b37bc68c424c5398122594364f2748"
format_stamp: "Formatted at 2026-08-09 13:01:43 on dist-test-slave-gp6n"
server_key: "99b9f0e94384928d235a618511ccb6aa"
server_key_iv: "afcb26e9464b5e63fcdc337a1ede43de"
server_key_version: "encryptionkey@0"
I20260809 13:01:43.832048 13398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "b9b37bc68c424c5398122594364f2748"
format_stamp: "Formatted at 2026-08-09 13:01:43 on dist-test-slave-gp6n"
server_key: "99b9f0e94384928d235a618511ccb6aa"
server_key_iv: "afcb26e9464b5e63fcdc337a1ede43de"
server_key_version: "encryptionkey@0"
I20260809 13:01:43.837249 13398 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:01:43.840871 13414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:43.842206 13398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 13:01:43.842391 13398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "b9b37bc68c424c5398122594364f2748"
format_stamp: "Formatted at 2026-08-09 13:01:43 on dist-test-slave-gp6n"
server_key: "99b9f0e94384928d235a618511ccb6aa"
server_key_iv: "afcb26e9464b5e63fcdc337a1ede43de"
server_key_version: "encryptionkey@0"
I20260809 13:01:43.842671 13398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:43.862543 13398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:43.866717 13398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:43.867009 13398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:43.886253 13398 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45809
I20260809 13:01:43.886405 13472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45809 every 8 connection(s)
I20260809 13:01:43.887698 13398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:01:43.891024 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 13398
I20260809 13:01:43.891232 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:01:43.891676 22445 external_mini_cluster.cc:1428] Setting key b393dac369aeb8a709704baf3be69c80
I20260809 13:01:43.891906 13473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:43.907920 13473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748: Bootstrap starting.
Aug 09 13:01:43 dist-test-slave-gp6n krb5kdc[13377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 13:01:43.912588 13473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:43.914132 13473 log.cc:824] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:43.917987 13473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748: No bootstrap required, opened a new log
I20260809 13:01:43.919871 13477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:43.893390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35603 (local address 127.0.0.1:45809)
0809 13:01:43.894166 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:43.894177 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:43.894216 (+ 39us) server_negotiation.cc:407] Connection header received
0809 13:01:43.896787 (+ 2571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:43.896808 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:43.897190 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:43.897514 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:43.899643 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:43.900958 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:43.903045 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:43.903750 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:43.903891 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:43.910236 (+ 6345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:43.910251 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:43.910283 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:43.910337 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:43.914793 (+ 4456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:43.915768 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:43.915778 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:43.915787 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:43.916017 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:43.916412 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:43.916417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:43.916420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:43.917172 (+ 752us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:43.917549 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:43.917869 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 13:01:43.918345 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":364,"threads_started":1}
I20260809 13:01:43.924188 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:01:43.930063 13473 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9b37bc68c424c5398122594364f2748" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45809 } }
I20260809 13:01:43.930644 13473 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:43.930799 13473 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9b37bc68c424c5398122594364f2748, State: Initialized, Role: FOLLOWER
I20260809 13:01:43.931480 13473 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9b37bc68c424c5398122594364f2748" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45809 } }
I20260809 13:01:43.931746 13473 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:01:43.931926 13473 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:01:43.932129 13473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:43.934729 13473 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9b37bc68c424c5398122594364f2748" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45809 } }
I20260809 13:01:43.935309 13473 leader_election.cc:302] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9b37bc68c424c5398122594364f2748; no voters:
I20260809 13:01:43.936039 13473 leader_election.cc:288] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:01:43.936147 13484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:43.938977 13473 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:01:43.943223 13484 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748 [term 1 LEADER]: Becoming Leader. State: Replica: b9b37bc68c424c5398122594364f2748, State: Running, Role: LEADER
I20260809 13:01:43.943950 13484 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9b37bc68c424c5398122594364f2748" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45809 } }
I20260809 13:01:43.944949 13477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:43.930060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58011 (local address 127.0.0.1:45809)
0809 13:01:43.930266 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:43.930284 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:43.930709 (+ 425us) server_negotiation.cc:407] Connection header received
0809 13:01:43.930899 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:43.930905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:43.931008 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:43.931112 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:43.932774 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:43.933567 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:43.934869 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:43.935399 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:43.935545 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:43.938300 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:43.938305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:43.938308 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:43.938339 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:43.941869 (+ 3530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:43.942586 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:43.942592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:43.942602 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:43.942770 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:43.943301 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:43.943304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:43.943306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:43.943728 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:43.944040 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:43.944353 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 13:01:43.944575 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 13:01:43.948894 13486 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9b37bc68c424c5398122594364f2748. Latest consensus state: current_term: 1 leader_uuid: "b9b37bc68c424c5398122594364f2748" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9b37bc68c424c5398122594364f2748" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45809 } } }
I20260809 13:01:43.948932 13485 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9b37bc68c424c5398122594364f2748" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9b37bc68c424c5398122594364f2748" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45809 } } }
I20260809 13:01:43.949440 13485 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:43.949440 13486 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:43.951417 13502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:01:43.956367 13502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:01:43.964839 13502 catalog_manager.cc:1347] Generated new cluster ID: 50c5dd5a6d204df1b66184189c6881a1
I20260809 13:01:43.964968 13502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:01:43.971127 13502 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:01:43.973181 13502 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:01:43.983636 13502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9b37bc68c424c5398122594364f2748: Generated new TSK 0
I20260809 13:01:43.984787 13502 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:01:43.991717 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 13398
I20260809 13:01:43.993042 13468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:44.084318 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 13398
2026-08-09T13:01:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2214 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:01:44 dist-test-slave-gp6n krb5kdc[13510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:01:44 dist-test-slave-gp6n krb5kdc[13510](info): set up 2 sockets
Aug 09 13:01:44 dist-test-slave-gp6n krb5kdc[13510](info): commencing operation
krb5kdc: starting...
W20260809 13:01:45.294252 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.002s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:01:45 dist-test-slave-gp6n krb5kdc[13510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:01:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:01:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:01:45.610201 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43183
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43183
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:45.949024 13525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:45.949396 13525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:45.949558 13525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:45.956753 13525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:01:45.956904 13525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:45.957015 13525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:45.957122 13525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:01:45.957203 13525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:01:45.967710 13525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:45.969074 13525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:45.970480 13525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:45.982798 13532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:45.982796 13533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:45.983043 13535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:45.983801 13525 server_base.cc:1034] running on GCE node
I20260809 13:01:45.984679 13525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:45.986096 13525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:45.987290 13525 hybrid_clock.cc:648] HybridClock initialized: now 1786280505987235 us; error 52 us; skew 500 ppm
I20260809 13:01:45.987664 13525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:45 dist-test-slave-gp6n krb5kdc[13510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280505, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:45.990906 13525 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:01:45.992419 13525 webserver.cc:460] Webserver started at http://127.0.0.1:33443/ using document root <none> and password file <none>
I20260809 13:01:45.993119 13525 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:45.993260 13525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:45.993582 13525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:45.996712 13525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "1cb18e8c0d0d44fb947ae03df3bac537"
format_stamp: "Formatted at 2026-08-09 13:01:45 on dist-test-slave-gp6n"
server_key: "1b16c7f2f840c9b1ab98ac5aa1d526b3"
server_key_iv: "cd29d30d0b4ad31bb820f6b8dd378cdf"
server_key_version: "encryptionkey@0"
I20260809 13:01:45.997521 13525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "1cb18e8c0d0d44fb947ae03df3bac537"
format_stamp: "Formatted at 2026-08-09 13:01:45 on dist-test-slave-gp6n"
server_key: "1b16c7f2f840c9b1ab98ac5aa1d526b3"
server_key_iv: "cd29d30d0b4ad31bb820f6b8dd378cdf"
server_key_version: "encryptionkey@0"
I20260809 13:01:46.002933 13525 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 13:01:46.006650 13542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:46.008281 13525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:01:46.008502 13525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "1cb18e8c0d0d44fb947ae03df3bac537"
format_stamp: "Formatted at 2026-08-09 13:01:45 on dist-test-slave-gp6n"
server_key: "1b16c7f2f840c9b1ab98ac5aa1d526b3"
server_key_iv: "cd29d30d0b4ad31bb820f6b8dd378cdf"
server_key_version: "encryptionkey@0"
I20260809 13:01:46.008719 13525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:46.041929 13525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:46.046280 13525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:46.046603 13525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:46.065452 13525 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43183
I20260809 13:01:46.065503 13600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43183 every 8 connection(s)
I20260809 13:01:46.066911 13525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:01:46.070461 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 13525
I20260809 13:01:46.070669 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:01:46.071111 13601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:46.071254 22445 external_mini_cluster.cc:1428] Setting key 313cedd8d26ae39b81b286708bff0c99
I20260809 13:01:46.085361 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:46.073096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36805 (local address 127.0.0.1:43183)
0809 13:01:46.073911 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:46.073923 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:46.073966 (+ 43us) server_negotiation.cc:407] Connection header received
0809 13:01:46.075946 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:46.075974 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:46.076364 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:46.076681 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:46.077351 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:46.078800 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:46.080815 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:46.081465 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:46.081588 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:46.083091 (+ 1503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:46.083098 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:46.083111 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:46.083680 (+ 569us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:46.083761 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:46.083785 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:46.083873 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:46.084130 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 13:01:46.084381 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":382,"threads_started":1}
I20260809 13:01:46.086386 13601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537: Bootstrap starting.
I20260809 13:01:46.088995 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:01:46.090189 13601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:46.091643 13601 log.cc:824] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:46.094555 13601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537: No bootstrap required, opened a new log
I20260809 13:01:46.101917 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:46.094945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59399 (local address 127.0.0.1:43183)
0809 13:01:46.095199 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:46.095210 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:46.095649 (+ 439us) server_negotiation.cc:407] Connection header received
0809 13:01:46.095827 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:46.095831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:46.095915 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:46.096008 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:46.096626 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:46.097395 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:46.099376 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:46.099846 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:46.099954 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:46.101145 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:46.101150 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:46.101156 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:46.101174 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:46.101216 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:46.101231 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:46.101314 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:46.101467 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 13:01:46.101544 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 13:01:46.104843 13601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cb18e8c0d0d44fb947ae03df3bac537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43183 } }
I20260809 13:01:46.105252 13601 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:46.105355 13601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb18e8c0d0d44fb947ae03df3bac537, State: Initialized, Role: FOLLOWER
I20260809 13:01:46.105996 13601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cb18e8c0d0d44fb947ae03df3bac537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43183 } }
I20260809 13:01:46.106241 13601 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:01:46.106388 13601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:01:46.106601 13601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:46.109305 13601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cb18e8c0d0d44fb947ae03df3bac537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43183 } }
I20260809 13:01:46.109803 13601 leader_election.cc:302] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cb18e8c0d0d44fb947ae03df3bac537; no voters:
I20260809 13:01:46.110513 13601 leader_election.cc:288] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:01:46.110664 13612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:46.111389 13612 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537 [term 1 LEADER]: Becoming Leader. State: Replica: 1cb18e8c0d0d44fb947ae03df3bac537, State: Running, Role: LEADER
I20260809 13:01:46.112056 13612 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cb18e8c0d0d44fb947ae03df3bac537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43183 } }
I20260809 13:01:46.112782 13601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:01:46.115314 13613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cb18e8c0d0d44fb947ae03df3bac537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cb18e8c0d0d44fb947ae03df3bac537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43183 } } }
I20260809 13:01:46.115398 13614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cb18e8c0d0d44fb947ae03df3bac537. Latest consensus state: current_term: 1 leader_uuid: "1cb18e8c0d0d44fb947ae03df3bac537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cb18e8c0d0d44fb947ae03df3bac537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43183 } } }
I20260809 13:01:46.115885 13614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:46.115890 13613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:46.117005 13621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:01:46.122527 13621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:01:46.131738 13621 catalog_manager.cc:1347] Generated new cluster ID: e2410541e43b4b3fa5472b46d5ce177a
I20260809 13:01:46.131870 13621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:01:46.145068 13621 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:01:46.146657 13621 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:01:46.158072 13621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cb18e8c0d0d44fb947ae03df3bac537: Generated new TSK 0
I20260809 13:01:46.159343 13621 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:01:46.175334 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 13525
W20260809 13:01:46.176644 13596 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36805
W20260809 13:01:46.177388 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 13525: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:01:46.177533 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 13525
2026-08-09T13:01:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2093 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:01:46 dist-test-slave-gp6n krb5kdc[13633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:01:46 dist-test-slave-gp6n krb5kdc[13633](info): set up 2 sockets
Aug 09 13:01:46 dist-test-slave-gp6n krb5kdc[13633](info): commencing operation
krb5kdc: starting...
W20260809 13:01:47.381037 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:01:47 dist-test-slave-gp6n krb5kdc[13633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:01:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:01:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:01:47.710158 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41321
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:48.060316 13649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:48.060613 13649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:48.060770 13649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:48.068234 13649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:01:48.068347 13649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:48.068449 13649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:48.068549 13649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:01:48.068629 13649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:01:48.078827 13649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:48.080214 13649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:48.081596 13649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:48.094017 13655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:48.094082 13656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:48.094099 13658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:48.095430 13649 server_base.cc:1034] running on GCE node
I20260809 13:01:48.096241 13649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:48.097790 13649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:48.099030 13649 hybrid_clock.cc:648] HybridClock initialized: now 1786280508098984 us; error 36 us; skew 500 ppm
I20260809 13:01:48.099427 13649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:48 dist-test-slave-gp6n krb5kdc[13633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280508, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:48.102734 13649 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:01:48.104238 13649 webserver.cc:460] Webserver started at http://127.0.0.1:39585/ using document root <none> and password file <none>
I20260809 13:01:48.104898 13649 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:48.105006 13649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:48.105258 13649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:48.108304 13649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "014a75de02b84b009ef569499950cffd"
format_stamp: "Formatted at 2026-08-09 13:01:48 on dist-test-slave-gp6n"
server_key: "73a79b5d8114ae24aeb77df1dc2a5894"
server_key_iv: "983dd9d4a5416cbe52fe0b02b46478c9"
server_key_version: "encryptionkey@0"
I20260809 13:01:48.109041 13649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "014a75de02b84b009ef569499950cffd"
format_stamp: "Formatted at 2026-08-09 13:01:48 on dist-test-slave-gp6n"
server_key: "73a79b5d8114ae24aeb77df1dc2a5894"
server_key_iv: "983dd9d4a5416cbe52fe0b02b46478c9"
server_key_version: "encryptionkey@0"
I20260809 13:01:48.114466 13649 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 13:01:48.117976 13665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:48.119580 13649 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 13:01:48.119778 13649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "014a75de02b84b009ef569499950cffd"
format_stamp: "Formatted at 2026-08-09 13:01:48 on dist-test-slave-gp6n"
server_key: "73a79b5d8114ae24aeb77df1dc2a5894"
server_key_iv: "983dd9d4a5416cbe52fe0b02b46478c9"
server_key_version: "encryptionkey@0"
I20260809 13:01:48.119999 13649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:48.125871 13649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:48.129845 13649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:48.130163 13649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:48.149091 13649 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41321
I20260809 13:01:48.149134 13728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41321 every 8 connection(s)
I20260809 13:01:48.150694 13649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:01:48.151314 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 13649
I20260809 13:01:48.151520 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:01:48.152130 22445 external_mini_cluster.cc:1428] Setting key 598db177ab3e840e849d57dbf60072be
I20260809 13:01:48.155507 13730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:48.161161 13733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:48.153968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44719 (local address 127.0.0.1:41321)
0809 13:01:48.154823 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:48.154840 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:48.154892 (+ 52us) server_negotiation.cc:407] Connection header received
0809 13:01:48.157217 (+ 2325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:48.157244 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:48.157681 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:48.158110 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:48.159171 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:48.159179 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:48.159200 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:48.159217 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:48.159330 (+ 113us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:48.159369 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:48.159579 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:48.159710 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 13:01:48.159921 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":377,"threads_started":1}
I20260809 13:01:48.165081 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:01:48.168774 13730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd: Bootstrap starting.
I20260809 13:01:48.172204 13733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:48.169492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41585 (local address 127.0.0.1:41321)
0809 13:01:48.169700 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:48.169710 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:48.170172 (+ 462us) server_negotiation.cc:407] Connection header received
0809 13:01:48.170350 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:48.170354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:48.170452 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:48.170558 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:48.171548 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:48.171552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:48.171556 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:48.171570 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:48.171608 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:48.171623 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:48.171712 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:48.171856 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 13:01:48.171903 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 13:01:48.172994 13730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:48.174340 13730 log.cc:824] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:48.177222 13730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd: No bootstrap required, opened a new log
I20260809 13:01:48.185292 13730 raft_consensus.cc:348] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014a75de02b84b009ef569499950cffd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41321 } }
I20260809 13:01:48.185665 13730 raft_consensus.cc:374] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:48.185742 13730 raft_consensus.cc:729] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 014a75de02b84b009ef569499950cffd, State: Initialized, Role: FOLLOWER
I20260809 13:01:48.186259 13730 consensus_queue.cc:260] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014a75de02b84b009ef569499950cffd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41321 } }
I20260809 13:01:48.186537 13730 raft_consensus.cc:388] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:01:48.186648 13730 raft_consensus.cc:482] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:01:48.186775 13730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:48.189360 13730 raft_consensus.cc:504] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014a75de02b84b009ef569499950cffd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41321 } }
I20260809 13:01:48.189823 13730 leader_election.cc:302] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 014a75de02b84b009ef569499950cffd; no voters:
I20260809 13:01:48.190388 13730 leader_election.cc:288] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:01:48.190557 13741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:48.191259 13741 raft_consensus.cc:686] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd [term 1 LEADER]: Becoming Leader. State: Replica: 014a75de02b84b009ef569499950cffd, State: Running, Role: LEADER
I20260809 13:01:48.191819 13741 consensus_queue.cc:237] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014a75de02b84b009ef569499950cffd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41321 } }
I20260809 13:01:48.192770 13730 sys_catalog.cc:564] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:01:48.194648 13743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 014a75de02b84b009ef569499950cffd. Latest consensus state: current_term: 1 leader_uuid: "014a75de02b84b009ef569499950cffd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014a75de02b84b009ef569499950cffd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41321 } } }
I20260809 13:01:48.194921 13742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "014a75de02b84b009ef569499950cffd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014a75de02b84b009ef569499950cffd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41321 } } }
I20260809 13:01:48.195211 13743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:48.195230 13742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:48.196950 13747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:01:48.201781 13747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:01:48.209925 13747 catalog_manager.cc:1347] Generated new cluster ID: 3a9c6085cbef4db9b702dbd1f793b0c6
I20260809 13:01:48.210050 13747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:01:48.232623 13747 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:01:48.233083 13747 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:01:48.239871 13747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 014a75de02b84b009ef569499950cffd: Generated new TSK 0
I20260809 13:01:48.240702 13747 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:01:48.250012 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 13649
W20260809 13:01:48.251392 13724 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44719
W20260809 13:01:48.252184 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 13649: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:01:48.252367 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 13649
2026-08-09T13:01:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2073 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:01:48 dist-test-slave-gp6n krb5kdc[13763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:01:48 dist-test-slave-gp6n krb5kdc[13763](info): set up 2 sockets
Aug 09 13:01:48 dist-test-slave-gp6n krb5kdc[13763](info): commencing operation
krb5kdc: starting...
W20260809 13:01:49.469135 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:01:49 dist-test-slave-gp6n krb5kdc[13763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:01:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:01:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:01:49.794188 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41803
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:50.149976 13776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:50.150303 13776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:50.150506 13776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:50.157887 13776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:01:50.158002 13776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:50.158093 13776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:50.158208 13776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:01:50.158285 13776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:01:50.168459 13776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:50.169909 13776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:50.171271 13776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:50.183612 13783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:50.183563 13784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:50.183643 13786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:50.184652 13776 server_base.cc:1034] running on GCE node
I20260809 13:01:50.185516 13776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:50.187125 13776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:50.188364 13776 hybrid_clock.cc:648] HybridClock initialized: now 1786280510188310 us; error 36 us; skew 500 ppm
I20260809 13:01:50.188773 13776 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:50 dist-test-slave-gp6n krb5kdc[13763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280510, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:50.197476 13776 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:01:50.199254 13776 webserver.cc:460] Webserver started at http://127.0.0.1:35573/ using document root <none> and password file <none>
I20260809 13:01:50.200033 13776 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:50.200173 13776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:50.200508 13776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:50.203802 13776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "74b4fb92cfc24c22bb3a6b99c619be95"
format_stamp: "Formatted at 2026-08-09 13:01:50 on dist-test-slave-gp6n"
server_key: "4f103f7dde71e30e1850e6b77c455dcd"
server_key_iv: "051f50a2eb8a702e1e6d6e1528188500"
server_key_version: "encryptionkey@0"
I20260809 13:01:50.204624 13776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "74b4fb92cfc24c22bb3a6b99c619be95"
format_stamp: "Formatted at 2026-08-09 13:01:50 on dist-test-slave-gp6n"
server_key: "4f103f7dde71e30e1850e6b77c455dcd"
server_key_iv: "051f50a2eb8a702e1e6d6e1528188500"
server_key_version: "encryptionkey@0"
I20260809 13:01:50.210234 13776 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 13:01:50.214018 13796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:50.215552 13776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:01:50.215749 13776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "74b4fb92cfc24c22bb3a6b99c619be95"
format_stamp: "Formatted at 2026-08-09 13:01:50 on dist-test-slave-gp6n"
server_key: "4f103f7dde71e30e1850e6b77c455dcd"
server_key_iv: "051f50a2eb8a702e1e6d6e1528188500"
server_key_version: "encryptionkey@0"
I20260809 13:01:50.215952 13776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:50.233708 13776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:50.238685 13776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:50.238988 13776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:50.258672 13776 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41803
I20260809 13:01:50.258715 13853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41803 every 8 connection(s)
I20260809 13:01:50.260046 13776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:01:50.263813 13854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:50.267865 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 13776
I20260809 13:01:50.268061 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:01:50.268635 22445 external_mini_cluster.cc:1428] Setting key 653a1557f45bc924327acc9d566f77e7
I20260809 13:01:50.276515 13854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95: Bootstrap starting.
I20260809 13:01:50.279816 13854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:50.281093 13854 log.cc:824] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95: Log is configured to *not* fsync() on all Append() calls
Aug 09 13:01:50 dist-test-slave-gp6n krb5kdc[13763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 13:01:50.284554 13854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95: No bootstrap required, opened a new log
I20260809 13:01:50.294376 13859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:50.270557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:42521 (local address 127.0.0.1:41803)
0809 13:01:50.271278 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:50.271290 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:50.271331 (+ 41us) server_negotiation.cc:407] Connection header received
0809 13:01:50.273100 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:50.273123 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:50.273546 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:50.273955 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:50.275487 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:50.276863 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:50.278554 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:50.279281 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:50.279513 (+ 232us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:50.284118 (+ 4605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:50.284130 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:50.284151 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:50.284197 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:50.288692 (+ 4495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:50.289807 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:50.289817 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:50.289826 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:50.290066 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:50.290636 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:50.290642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:50.290645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:50.291521 (+ 876us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:50.291914 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:50.292323 (+ 409us) server_negotiation.cc:299] Negotiation successful
0809 13:01:50.292781 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":307,"threads_started":1}
I20260809 13:01:50.296288 13854 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b4fb92cfc24c22bb3a6b99c619be95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41803 } }
I20260809 13:01:50.296707 13854 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:50.296835 13854 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74b4fb92cfc24c22bb3a6b99c619be95, State: Initialized, Role: FOLLOWER
I20260809 13:01:50.297556 13854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b4fb92cfc24c22bb3a6b99c619be95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41803 } }
I20260809 13:01:50.297847 13854 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:01:50.298005 13854 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:01:50.298190 13854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:50.299197 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:01:50.300813 13854 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b4fb92cfc24c22bb3a6b99c619be95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41803 } }
I20260809 13:01:50.301380 13854 leader_election.cc:302] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74b4fb92cfc24c22bb3a6b99c619be95; no voters:
I20260809 13:01:50.302151 13854 leader_election.cc:288] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:01:50.302270 13862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:50.303746 13862 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95 [term 1 LEADER]: Becoming Leader. State: Replica: 74b4fb92cfc24c22bb3a6b99c619be95, State: Running, Role: LEADER
I20260809 13:01:50.304432 13862 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b4fb92cfc24c22bb3a6b99c619be95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41803 } }
I20260809 13:01:50.305047 13854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:01:50.311851 13865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74b4fb92cfc24c22bb3a6b99c619be95. Latest consensus state: current_term: 1 leader_uuid: "74b4fb92cfc24c22bb3a6b99c619be95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b4fb92cfc24c22bb3a6b99c619be95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41803 } } }
I20260809 13:01:50.312390 13865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:50.313127 13864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74b4fb92cfc24c22bb3a6b99c619be95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b4fb92cfc24c22bb3a6b99c619be95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41803 } } }
I20260809 13:01:50.313506 13864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:50.327716 13859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:50.306007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:54035 (local address 127.0.0.1:41803)
0809 13:01:50.306236 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:50.306246 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:50.306923 (+ 677us) server_negotiation.cc:407] Connection header received
0809 13:01:50.307150 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:50.307155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:50.307270 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:50.307361 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:50.308944 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:50.309754 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:50.311758 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:50.312452 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:50.312574 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:50.317046 (+ 4472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:50.317057 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:50.317065 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:50.317121 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:50.322194 (+ 5073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:50.323812 (+ 1618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:50.323821 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:50.323827 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:50.324090 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:50.324861 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:50.324884 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:50.324890 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:50.325516 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:50.325887 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:50.326900 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0809 13:01:50.327203 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 13:01:50.334525 13872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:01:50.341534 13872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:01:50.353166 13872 catalog_manager.cc:1347] Generated new cluster ID: 18e1114092094c0aa6ea773fa7188629
I20260809 13:01:50.353302 13872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:01:50.368932 13872 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:01:50.370958 13872 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:01:50.383998 13872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74b4fb92cfc24c22bb3a6b99c619be95: Generated new TSK 0
I20260809 13:01:50.384904 13872 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:01:50.400089 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 13776
I20260809 13:01:50.401353 13849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:50.497116 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 13776
W20260809 13:01:50.518049 13761 connection.cc:535] client connection to 127.0.0.1:41803 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41803): Connection reset by peer (error 104)
2026-08-09T13:01:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2245 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:01:50 dist-test-slave-gp6n krb5kdc[13893](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:01:50 dist-test-slave-gp6n krb5kdc[13893](info): set up 2 sockets
Aug 09 13:01:50 dist-test-slave-gp6n krb5kdc[13893](info): commencing operation
krb5kdc: starting...
W20260809 13:01:51.692673 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:01:51 dist-test-slave-gp6n krb5kdc[13893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:01:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:01:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:01:52.014526 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35823
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:52.366753 13908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:52.367105 13908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:52.367270 13908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:52.374578 13908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:01:52.374723 13908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:52.374823 13908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:52.374933 13908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:01:52.375011 13908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:01:52.385216 13908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:52.386791 13908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:52.388162 13908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:52.400482 13913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:52.400434 13914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:52.400524 13916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:52.401422 13908 server_base.cc:1034] running on GCE node
I20260809 13:01:52.402202 13908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:52.403690 13908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:52.404915 13908 hybrid_clock.cc:648] HybridClock initialized: now 1786280512404883 us; error 38 us; skew 500 ppm
I20260809 13:01:52.405277 13908 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:52 dist-test-slave-gp6n krb5kdc[13893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280512, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:52.408666 13908 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:01:52.410252 13908 webserver.cc:460] Webserver started at http://127.0.0.1:44845/ using document root <none> and password file <none>
I20260809 13:01:52.411000 13908 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:52.411149 13908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:52.411470 13908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:52.415107 13908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "49603b6e61cf4bc99d8b0fa9c7fa71cc"
format_stamp: "Formatted at 2026-08-09 13:01:52 on dist-test-slave-gp6n"
server_key: "06f4d1b7fb1c7e19f32c46fb976f442b"
server_key_iv: "e71d58e875a199ea4ed27f979c6af819"
server_key_version: "encryptionkey@0"
I20260809 13:01:52.416296 13908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "49603b6e61cf4bc99d8b0fa9c7fa71cc"
format_stamp: "Formatted at 2026-08-09 13:01:52 on dist-test-slave-gp6n"
server_key: "06f4d1b7fb1c7e19f32c46fb976f442b"
server_key_iv: "e71d58e875a199ea4ed27f979c6af819"
server_key_version: "encryptionkey@0"
I20260809 13:01:52.421923 13908 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 13:01:52.425472 13923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:52.426863 13908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 13:01:52.427047 13908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "49603b6e61cf4bc99d8b0fa9c7fa71cc"
format_stamp: "Formatted at 2026-08-09 13:01:52 on dist-test-slave-gp6n"
server_key: "06f4d1b7fb1c7e19f32c46fb976f442b"
server_key_iv: "e71d58e875a199ea4ed27f979c6af819"
server_key_version: "encryptionkey@0"
I20260809 13:01:52.427243 13908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:52.440632 13908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:52.445109 13908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:52.445439 13908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:52.465360 13908 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35823
I20260809 13:01:52.465389 13983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35823 every 8 connection(s)
I20260809 13:01:52.466887 13908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:01:52.467535 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 13908
I20260809 13:01:52.467779 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:01:52.468387 22445 external_mini_cluster.cc:1428] Setting key 2cdefb9dd1365433d9066cd1bd456e01
I20260809 13:01:52.472051 13986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:52.483292 13991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:52.470288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:46681 (local address 127.0.0.1:35823)
0809 13:01:52.471436 (+ 1148us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:52.471450 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:52.471500 (+ 50us) server_negotiation.cc:407] Connection header received
0809 13:01:52.473416 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:52.473442 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:52.473893 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:52.474272 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:52.474974 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:52.476576 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:52.478571 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:52.479275 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:52.479421 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:52.480787 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:52.480795 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:52.480809 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:52.481391 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:52.481474 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:52.481503 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:52.481636 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:52.481961 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 13:01:52.482182 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":895,"thread_start_us":423,"threads_started":1}
I20260809 13:01:52.485277 13986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc: Bootstrap starting.
I20260809 13:01:52.487603 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:01:52.488689 13986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:52.489744 13986 log.cc:824] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:52.492692 13986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc: No bootstrap required, opened a new log
I20260809 13:01:52.498579 13991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:52.492148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:42753 (local address 127.0.0.1:35823)
0809 13:01:52.492332 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:52.492338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:52.492815 (+ 477us) server_negotiation.cc:407] Connection header received
0809 13:01:52.493030 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:52.493034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:52.493103 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:52.493210 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:52.493711 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:52.494690 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:52.495983 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:52.496460 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:52.496595 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:52.497786 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:52.497794 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:52.497798 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:52.497816 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:52.497855 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:52.497870 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:52.498003 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:52.498190 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 13:01:52.498244 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 13:01:52.501173 13986 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49603b6e61cf4bc99d8b0fa9c7fa71cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35823 } }
I20260809 13:01:52.501585 13986 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:52.501688 13986 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49603b6e61cf4bc99d8b0fa9c7fa71cc, State: Initialized, Role: FOLLOWER
I20260809 13:01:52.502359 13986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49603b6e61cf4bc99d8b0fa9c7fa71cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35823 } }
I20260809 13:01:52.502679 13986 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:01:52.502835 13986 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:01:52.503021 13986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:52.505678 13986 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49603b6e61cf4bc99d8b0fa9c7fa71cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35823 } }
I20260809 13:01:52.506173 13986 leader_election.cc:302] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49603b6e61cf4bc99d8b0fa9c7fa71cc; no voters:
I20260809 13:01:52.506852 13986 leader_election.cc:288] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:01:52.506955 13998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:52.508363 13998 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc [term 1 LEADER]: Becoming Leader. State: Replica: 49603b6e61cf4bc99d8b0fa9c7fa71cc, State: Running, Role: LEADER
I20260809 13:01:52.508919 13998 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49603b6e61cf4bc99d8b0fa9c7fa71cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35823 } }
I20260809 13:01:52.509174 13986 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:01:52.512259 14000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49603b6e61cf4bc99d8b0fa9c7fa71cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49603b6e61cf4bc99d8b0fa9c7fa71cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35823 } } }
I20260809 13:01:52.512686 14000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:52.513123 14001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49603b6e61cf4bc99d8b0fa9c7fa71cc. Latest consensus state: current_term: 1 leader_uuid: "49603b6e61cf4bc99d8b0fa9c7fa71cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49603b6e61cf4bc99d8b0fa9c7fa71cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35823 } } }
I20260809 13:01:52.513440 14001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:52.513720 14008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:01:52.522289 14008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:01:52.531234 14008 catalog_manager.cc:1347] Generated new cluster ID: 6d359d4c1c6f44bb8a8d058b7878d228
I20260809 13:01:52.531391 14008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:01:52.548303 14008 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:01:52.549672 14008 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:01:52.562579 14008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49603b6e61cf4bc99d8b0fa9c7fa71cc: Generated new TSK 0
I20260809 13:01:52.563416 14008 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:01:52.571004 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 13908
W20260809 13:01:52.572260 13979 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.44:46681
W20260809 13:01:52.573061 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 13908: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:01:52.573222 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 13908
W20260809 13:01:52.593004 13890 connection.cc:535] client connection to 127.0.0.1:35823 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35823): Connection reset by peer (error 104)
2026-08-09T13:01:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2075 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:01:52 dist-test-slave-gp6n krb5kdc[14022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:01:52 dist-test-slave-gp6n krb5kdc[14022](info): set up 2 sockets
Aug 09 13:01:52 dist-test-slave-gp6n krb5kdc[14022](info): commencing operation
krb5kdc: starting...
W20260809 13:01:53.793170 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.008s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:01:53 dist-test-slave-gp6n krb5kdc[14022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:01:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:01:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:01:54.124390 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:54.485281 14039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:54.485592 14039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:54.485773 14039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:54.493191 14039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:01:54.493342 14039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:54.493464 14039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:54.493566 14039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:01:54.493652 14039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:01:54.503881 14039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:54.505167 14039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:54.506575 14039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:54.518540 14050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:54.518610 14046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:54.518570 14047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:54.519554 14039 server_base.cc:1034] running on GCE node
I20260809 13:01:54.520360 14039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:54.521788 14039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:54.523054 14039 hybrid_clock.cc:648] HybridClock initialized: now 1786280514523006 us; error 56 us; skew 500 ppm
I20260809 13:01:54.523458 14039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:54 dist-test-slave-gp6n krb5kdc[14022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280514, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:54.526858 14039 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:01:54.528394 14039 webserver.cc:460] Webserver started at http://127.0.0.1:38241/ using document root <none> and password file <none>
I20260809 13:01:54.529083 14039 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:54.529232 14039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:54.529560 14039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:54.532603 14039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "448c9dd20ad646d68409075ec403d8b5"
format_stamp: "Formatted at 2026-08-09 13:01:54 on dist-test-slave-gp6n"
server_key: "3937b5c299a3723169f80e4e0f39171e"
server_key_iv: "89663da533f1e03bc81c6b5f04ab517b"
server_key_version: "encryptionkey@0"
I20260809 13:01:54.533412 14039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "448c9dd20ad646d68409075ec403d8b5"
format_stamp: "Formatted at 2026-08-09 13:01:54 on dist-test-slave-gp6n"
server_key: "3937b5c299a3723169f80e4e0f39171e"
server_key_iv: "89663da533f1e03bc81c6b5f04ab517b"
server_key_version: "encryptionkey@0"
I20260809 13:01:54.538901 14039 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 13:01:54.542335 14060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:54.543725 14039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:01:54.543915 14039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "448c9dd20ad646d68409075ec403d8b5"
format_stamp: "Formatted at 2026-08-09 13:01:54 on dist-test-slave-gp6n"
server_key: "3937b5c299a3723169f80e4e0f39171e"
server_key_iv: "89663da533f1e03bc81c6b5f04ab517b"
server_key_version: "encryptionkey@0"
I20260809 13:01:54.544131 14039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:54.550060 14039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:54.554595 14039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:54.554895 14039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:54.573823 14039 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35967
I20260809 13:01:54.573873 14119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35967 every 8 connection(s)
I20260809 13:01:54.575230 14039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:01:54.577275 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 14039
I20260809 13:01:54.577463 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:01:54.578020 22445 external_mini_cluster.cc:1428] Setting key 131d9fe8b389581b43d2246425133d34
I20260809 13:01:54.579754 14120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:54.585701 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:54.579810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:54245 (local address 127.0.0.1:35967)
0809 13:01:54.580674 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:54.580686 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:54.580728 (+ 42us) server_negotiation.cc:407] Connection header received
0809 13:01:54.582290 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:54.582310 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:54.582721 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:54.583054 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:54.584170 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:54.584176 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:54.584187 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:54.584200 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:54.584277 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:54.584295 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:54.584466 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:54.584565 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 13:01:54.584772 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":367,"threads_started":1}
I20260809 13:01:54.589322 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:01:54.593111 14120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5: Bootstrap starting.
I20260809 13:01:54.597179 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:54.594830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:51101 (local address 127.0.0.1:35967)
0809 13:01:54.595028 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:54.595034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:54.595338 (+ 304us) server_negotiation.cc:407] Connection header received
0809 13:01:54.595498 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:54.595501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:54.595546 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:54.595621 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:54.596536 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:54.596539 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:54.596542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:54.596556 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:54.596590 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:54.596603 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:54.596687 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:54.596827 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 13:01:54.596874 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 13:01:54.597204 14120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:54.598335 14120 log.cc:824] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:54.600893 14120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5: No bootstrap required, opened a new log
I20260809 13:01:54.608021 14120 raft_consensus.cc:348] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448c9dd20ad646d68409075ec403d8b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35967 } }
I20260809 13:01:54.608398 14120 raft_consensus.cc:374] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:54.608501 14120 raft_consensus.cc:729] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448c9dd20ad646d68409075ec403d8b5, State: Initialized, Role: FOLLOWER
I20260809 13:01:54.609058 14120 consensus_queue.cc:260] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448c9dd20ad646d68409075ec403d8b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35967 } }
I20260809 13:01:54.609313 14120 raft_consensus.cc:388] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:01:54.609478 14120 raft_consensus.cc:482] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:01:54.609694 14120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:54.612021 14120 raft_consensus.cc:504] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448c9dd20ad646d68409075ec403d8b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35967 } }
I20260809 13:01:54.612505 14120 leader_election.cc:302] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 448c9dd20ad646d68409075ec403d8b5; no voters:
I20260809 13:01:54.613062 14120 leader_election.cc:288] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:01:54.613193 14130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:54.613927 14130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5 [term 1 LEADER]: Becoming Leader. State: Replica: 448c9dd20ad646d68409075ec403d8b5, State: Running, Role: LEADER
I20260809 13:01:54.614521 14130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448c9dd20ad646d68409075ec403d8b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35967 } }
I20260809 13:01:54.615268 14120 sys_catalog.cc:564] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:01:54.617146 14131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "448c9dd20ad646d68409075ec403d8b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448c9dd20ad646d68409075ec403d8b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35967 } } }
I20260809 13:01:54.617497 14131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:54.618369 14132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 448c9dd20ad646d68409075ec403d8b5. Latest consensus state: current_term: 1 leader_uuid: "448c9dd20ad646d68409075ec403d8b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448c9dd20ad646d68409075ec403d8b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35967 } } }
I20260809 13:01:54.618775 14132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:54.620493 14136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:01:54.625533 14136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:01:54.634851 14136 catalog_manager.cc:1347] Generated new cluster ID: dbe0366ef0f649a684bf171c685ae707
I20260809 13:01:54.634992 14136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:01:54.648272 14136 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:01:54.648751 14136 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:01:54.658607 14136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 448c9dd20ad646d68409075ec403d8b5: Generated new TSK 0
I20260809 13:01:54.659411 14136 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:01:54.666208 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 14039
W20260809 13:01:54.667393 14114 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.44:54245
W20260809 13:01:54.668222 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 14039: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:01:54.668430 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 14039
2026-08-09T13:01:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2096 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:01:54 dist-test-slave-gp6n krb5kdc[14155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:01:54 dist-test-slave-gp6n krb5kdc[14155](info): set up 2 sockets
Aug 09 13:01:54 dist-test-slave-gp6n krb5kdc[14155](info): commencing operation
krb5kdc: starting...
W20260809 13:01:55.862478 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:01:56 dist-test-slave-gp6n krb5kdc[14155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:01:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:01:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:01:56.191285 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39129
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:56.547811 14169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:56.548094 14169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:56.548242 14169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:56.555603 14169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:01:56.555717 14169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:56.555792 14169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:56.555892 14169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:01:56.555968 14169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:01:56.565845 14169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39129
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:56.567211 14169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:56.568560 14169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:56.581110 14175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:56.581025 14176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:56.581079 14178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:56.581897 14169 server_base.cc:1034] running on GCE node
I20260809 13:01:56.582769 14169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:56.584232 14169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:56.585469 14169 hybrid_clock.cc:648] HybridClock initialized: now 1786280516585404 us; error 72 us; skew 500 ppm
I20260809 13:01:56.585860 14169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:56 dist-test-slave-gp6n krb5kdc[14155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280516, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:56.593405 14169 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:01:56.594928 14169 webserver.cc:460] Webserver started at http://127.0.0.1:44515/ using document root <none> and password file <none>
I20260809 13:01:56.595587 14169 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:56.595690 14169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:56.595942 14169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:56.598937 14169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "b2bec05f0ff04e1eb8cb62b74be6d550"
format_stamp: "Formatted at 2026-08-09 13:01:56 on dist-test-slave-gp6n"
server_key: "dfc8d7ab1dfa726d24bd45d8cf4d1411"
server_key_iv: "49978a107382a3f6d68bcda3807d6843"
server_key_version: "encryptionkey@0"
I20260809 13:01:56.599677 14169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "b2bec05f0ff04e1eb8cb62b74be6d550"
format_stamp: "Formatted at 2026-08-09 13:01:56 on dist-test-slave-gp6n"
server_key: "dfc8d7ab1dfa726d24bd45d8cf4d1411"
server_key_iv: "49978a107382a3f6d68bcda3807d6843"
server_key_version: "encryptionkey@0"
I20260809 13:01:56.604686 14169 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:01:56.608145 14186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:56.609459 14169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:01:56.609659 14169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "b2bec05f0ff04e1eb8cb62b74be6d550"
format_stamp: "Formatted at 2026-08-09 13:01:56 on dist-test-slave-gp6n"
server_key: "dfc8d7ab1dfa726d24bd45d8cf4d1411"
server_key_iv: "49978a107382a3f6d68bcda3807d6843"
server_key_version: "encryptionkey@0"
I20260809 13:01:56.609861 14169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:56.644085 14169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:56.648490 14169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:56.648814 14169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:56.667914 14169 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39129
I20260809 13:01:56.667963 14242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39129 every 8 connection(s)
I20260809 13:01:56.669287 14169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:01:56.672979 14243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:56.673928 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 14169
I20260809 13:01:56.674124 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:01:56.674718 22445 external_mini_cluster.cc:1428] Setting key f5e2fd8137d058470e976ff2e5673e3b
I20260809 13:01:56.687237 14243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550: Bootstrap starting.
Aug 09 13:01:56 dist-test-slave-gp6n krb5kdc[14155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 13:01:56.691098 14243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:56.692353 14243 log.cc:824] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:56.695503 14243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550: No bootstrap required, opened a new log
I20260809 13:01:56.698949 14246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:56.676600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:40187 (local address 127.0.0.1:39129)
0809 13:01:56.677390 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:56.677401 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:56.677441 (+ 40us) server_negotiation.cc:407] Connection header received
0809 13:01:56.679285 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:56.679309 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:56.679703 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:56.680026 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:56.681769 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:56.683137 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:56.684977 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:56.685632 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:56.685771 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:56.690979 (+ 5208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:56.690987 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:56.691005 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:56.691041 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:56.694697 (+ 3656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:56.695517 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:56.695531 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:56.695537 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:56.695751 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:56.696223 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:56.696229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:56.696232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:56.696741 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:56.697108 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:56.697564 (+ 456us) server_negotiation.cc:299] Negotiation successful
0809 13:01:56.697898 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":367,"threads_started":1}
I20260809 13:01:56.702839 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:01:56.703732 14243 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2bec05f0ff04e1eb8cb62b74be6d550" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39129 } }
I20260809 13:01:56.704087 14243 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:56.704188 14243 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2bec05f0ff04e1eb8cb62b74be6d550, State: Initialized, Role: FOLLOWER
I20260809 13:01:56.704794 14243 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2bec05f0ff04e1eb8cb62b74be6d550" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39129 } }
I20260809 13:01:56.705040 14243 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:01:56.705196 14243 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:01:56.705464 14243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:56.708981 14243 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2bec05f0ff04e1eb8cb62b74be6d550" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39129 } }
I20260809 13:01:56.709561 14243 leader_election.cc:302] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2bec05f0ff04e1eb8cb62b74be6d550; no voters:
I20260809 13:01:56.710240 14243 leader_election.cc:288] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:01:56.710337 14254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:56.712957 14243 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:01:56.714620 14254 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550 [term 1 LEADER]: Becoming Leader. State: Replica: b2bec05f0ff04e1eb8cb62b74be6d550, State: Running, Role: LEADER
I20260809 13:01:56.715395 14254 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2bec05f0ff04e1eb8cb62b74be6d550" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39129 } }
I20260809 13:01:56.718839 14255 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2bec05f0ff04e1eb8cb62b74be6d550" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2bec05f0ff04e1eb8cb62b74be6d550" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39129 } } }
I20260809 13:01:56.719627 14255 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:56.718892 14256 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2bec05f0ff04e1eb8cb62b74be6d550. Latest consensus state: current_term: 1 leader_uuid: "b2bec05f0ff04e1eb8cb62b74be6d550" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2bec05f0ff04e1eb8cb62b74be6d550" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39129 } } }
I20260809 13:01:56.720346 14256 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:56.728590 14246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:56.707356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:38705 (local address 127.0.0.1:39129)
0809 13:01:56.707508 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:56.707514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:56.709470 (+ 1956us) server_negotiation.cc:407] Connection header received
0809 13:01:56.709691 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:56.709695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:56.709779 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:56.709904 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:56.711790 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:56.712698 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:56.714268 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:56.715296 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:56.715468 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:56.719358 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:56.719370 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:56.719378 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:01:56.719428 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:56.723680 (+ 4252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:56.724998 (+ 1318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:56.725007 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:56.725013 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:56.725221 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:01:56.725964 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:01:56.725972 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:01:56.725977 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:01:56.726580 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:56.726881 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:56.727800 (+ 919us) server_negotiation.cc:299] Negotiation successful
0809 13:01:56.728049 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":51}
I20260809 13:01:56.731989 14265 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:01:56.737267 14265 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:01:56.746788 14265 catalog_manager.cc:1347] Generated new cluster ID: bd5ac83ef91546b79011f79165590cbe
I20260809 13:01:56.746927 14265 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:01:56.766564 14265 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:01:56.767905 14265 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:01:56.777417 14265 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2bec05f0ff04e1eb8cb62b74be6d550: Generated new TSK 0
I20260809 13:01:56.778339 14265 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:01:56.792496 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 14169
I20260809 13:01:56.794092 14238 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:01:56.889335 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 14169
W20260809 13:01:56.910305 14153 connection.cc:535] client connection to 127.0.0.1:39129 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39129): Connection reset by peer (error 104)
2026-08-09T13:01:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2223 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:01:57 dist-test-slave-gp6n krb5kdc[14277](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:01:57 dist-test-slave-gp6n krb5kdc[14277](info): set up 2 sockets
Aug 09 13:01:57 dist-test-slave-gp6n krb5kdc[14277](info): commencing operation
krb5kdc: starting...
W20260809 13:01:58.098810 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.006s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:01:58 dist-test-slave-gp6n krb5kdc[14277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:01:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:01:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:01:58.430608 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39125
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:01:58.785620 14292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:01:58.785898 14292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:01:58.786067 14292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:01:58.793488 14292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:01:58.793599 14292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:01:58.793711 14292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:01:58.793845 14292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:01:58.793912 14292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:01:58.804092 14292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39125
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:01:58.805411 14292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:01:58.806804 14292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:01:58.819382 14297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:58.819365 14301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:01:58.819343 14298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:01:58.820014 14292 server_base.cc:1034] running on GCE node
I20260809 13:01:58.820837 14292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:01:58.822359 14292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:01:58.823633 14292 hybrid_clock.cc:648] HybridClock initialized: now 1786280518823569 us; error 49 us; skew 500 ppm
I20260809 13:01:58.824019 14292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:01:58 dist-test-slave-gp6n krb5kdc[14277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280518, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:01:58.827355 14292 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:01:58.828866 14292 webserver.cc:460] Webserver started at http://127.0.0.1:34083/ using document root <none> and password file <none>
I20260809 13:01:58.829596 14292 fs_manager.cc:359] Metadata directory not provided
I20260809 13:01:58.829749 14292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:01:58.830062 14292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:01:58.833233 14292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "a214ad8e4d224fe786244bf33000de54"
format_stamp: "Formatted at 2026-08-09 13:01:58 on dist-test-slave-gp6n"
server_key: "1d25ff6b53a007fa79325b861c7a386d"
server_key_iv: "b34a38af7352d025a55c32898b0d99ee"
server_key_version: "encryptionkey@0"
I20260809 13:01:58.833994 14292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "a214ad8e4d224fe786244bf33000de54"
format_stamp: "Formatted at 2026-08-09 13:01:58 on dist-test-slave-gp6n"
server_key: "1d25ff6b53a007fa79325b861c7a386d"
server_key_iv: "b34a38af7352d025a55c32898b0d99ee"
server_key_version: "encryptionkey@0"
I20260809 13:01:58.839294 14292 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 13:01:58.842630 14309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:01:58.843997 14292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:01:58.844184 14292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "a214ad8e4d224fe786244bf33000de54"
format_stamp: "Formatted at 2026-08-09 13:01:58 on dist-test-slave-gp6n"
server_key: "1d25ff6b53a007fa79325b861c7a386d"
server_key_iv: "b34a38af7352d025a55c32898b0d99ee"
server_key_version: "encryptionkey@0"
I20260809 13:01:58.844385 14292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:01:58.863932 14292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:01:58.868579 14292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:01:58.868891 14292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:01:58.888569 14292 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39125
I20260809 13:01:58.888628 14366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39125 every 8 connection(s)
I20260809 13:01:58.890084 14292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:01:58.893983 14367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:01:58.894310 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 14292
I20260809 13:01:58.894563 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:01:58.895092 22445 external_mini_cluster.cc:1428] Setting key 370fd541798a2dd0531871ac36501247
I20260809 13:01:58.908268 14367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54: Bootstrap starting.
I20260809 13:01:58.910478 14370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:58.897296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:32855 (local address 127.0.0.1:39125)
0809 13:01:58.898240 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:58.898252 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:58.898312 (+ 60us) server_negotiation.cc:407] Connection header received
0809 13:01:58.900268 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:58.900297 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:58.900725 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:58.901075 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:58.901900 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:58.903450 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:58.905542 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:58.906260 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:58.906480 (+ 220us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:58.907782 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:58.907791 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:58.907805 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:58.908396 (+ 591us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:58.908534 (+ 138us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:58.908577 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:58.908720 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:58.909173 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 13:01:58.909387 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":432,"threads_started":1}
I20260809 13:01:58.911986 14367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54: Neither blocks nor log segments found. Creating new log.
I20260809 13:01:58.913128 14367 log.cc:824] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54: Log is configured to *not* fsync() on all Append() calls
I20260809 13:01:58.915210 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:01:58.916567 14367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54: No bootstrap required, opened a new log
I20260809 13:01:58.925702 14367 raft_consensus.cc:348] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a214ad8e4d224fe786244bf33000de54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39125 } }
I20260809 13:01:58.926132 14367 raft_consensus.cc:374] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:01:58.926273 14367 raft_consensus.cc:729] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a214ad8e4d224fe786244bf33000de54, State: Initialized, Role: FOLLOWER
I20260809 13:01:58.927078 14370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:01:58.920046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:42773 (local address 127.0.0.1:39125)
0809 13:01:58.920226 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 13:01:58.920235 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:01:58.920676 (+ 441us) server_negotiation.cc:407] Connection header received
0809 13:01:58.920859 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:01:58.920865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:01:58.920946 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:01:58.921074 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:01:58.921683 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:58.922907 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:58.924270 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:01:58.924966 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:01:58.925087 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:01:58.926222 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:01:58.926227 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:01:58.926231 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:01:58.926273 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:01:58.926357 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:01:58.926395 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:01:58.926532 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 13:01:58.926732 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 13:01:58.926788 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 13:01:58.927028 14367 consensus_queue.cc:260] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a214ad8e4d224fe786244bf33000de54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39125 } }
I20260809 13:01:58.927421 14367 raft_consensus.cc:388] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:01:58.927600 14367 raft_consensus.cc:482] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:01:58.927783 14367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:01:58.930605 14367 raft_consensus.cc:504] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a214ad8e4d224fe786244bf33000de54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39125 } }
I20260809 13:01:58.931233 14367 leader_election.cc:302] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a214ad8e4d224fe786244bf33000de54; no voters:
I20260809 13:01:58.931875 14367 leader_election.cc:288] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:01:58.932036 14377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:01:58.932792 14377 raft_consensus.cc:686] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54 [term 1 LEADER]: Becoming Leader. State: Replica: a214ad8e4d224fe786244bf33000de54, State: Running, Role: LEADER
I20260809 13:01:58.933406 14377 consensus_queue.cc:237] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a214ad8e4d224fe786244bf33000de54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39125 } }
I20260809 13:01:58.934098 14367 sys_catalog.cc:564] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:01:58.936291 14379 sys_catalog.cc:455] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a214ad8e4d224fe786244bf33000de54. Latest consensus state: current_term: 1 leader_uuid: "a214ad8e4d224fe786244bf33000de54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a214ad8e4d224fe786244bf33000de54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39125 } } }
I20260809 13:01:58.936661 14379 sys_catalog.cc:458] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:58.937175 14378 sys_catalog.cc:455] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a214ad8e4d224fe786244bf33000de54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a214ad8e4d224fe786244bf33000de54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39125 } } }
I20260809 13:01:58.937479 14378 sys_catalog.cc:458] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54 [sys.catalog]: This master's current role is: LEADER
I20260809 13:01:58.939301 14383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:01:58.945950 14383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:01:58.961076 14383 catalog_manager.cc:1347] Generated new cluster ID: 7a9c9225c191451baffae1a7b2a1a007
I20260809 13:01:58.961202 14383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:01:58.973685 14383 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:01:58.975603 14383 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:01:58.991608 14383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a214ad8e4d224fe786244bf33000de54: Generated new TSK 0
I20260809 13:01:58.992796 14383 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:01:59.003039 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 14292
W20260809 13:01:59.004163 14362 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.44:32855
W20260809 13:01:59.004949 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 14292: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:01:59.005108 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 14292
W20260809 13:01:59.023139 14275 connection.cc:535] client connection to 127.0.0.1:39125 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39125): Connection reset by peer (error 104)
2026-08-09T13:01:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2109 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:01:59 dist-test-slave-gp6n krb5kdc[14399](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:01:59 dist-test-slave-gp6n krb5kdc[14399](info): set up 2 sockets
Aug 09 13:01:59 dist-test-slave-gp6n krb5kdc[14399](info): commencing operation
krb5kdc: starting...
W20260809 13:02:00.220824 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.002s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:02:00 dist-test-slave-gp6n krb5kdc[14399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:02:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:02:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:02:00.546531 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38525
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:02:00.894213 14414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:02:00.894555 14414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:02:00.894766 14414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:02:00.902216 14414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:02:00.902361 14414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:02:00.902480 14414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:02:00.902597 14414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:02:00.902681 14414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:02:00.912693 14414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38525
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:02:00.914054 14414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:02:00.915444 14414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:02:00.927716 14419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:00.927743 14420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:00.927820 14422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:02:00.928695 14414 server_base.cc:1034] running on GCE node
I20260809 13:02:00.929607 14414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:02:00.931272 14414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:02:00.932542 14414 hybrid_clock.cc:648] HybridClock initialized: now 1786280520932502 us; error 34 us; skew 500 ppm
I20260809 13:02:00.932940 14414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:02:00 dist-test-slave-gp6n krb5kdc[14399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280520, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:02:00.936389 14414 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:02:00.937899 14414 webserver.cc:460] Webserver started at http://127.0.0.1:38729/ using document root <none> and password file <none>
I20260809 13:02:00.938702 14414 fs_manager.cc:359] Metadata directory not provided
I20260809 13:02:00.938849 14414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:02:00.939169 14414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:02:00.942292 14414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "665204fe600542d5b310db048f57f8ec"
format_stamp: "Formatted at 2026-08-09 13:02:00 on dist-test-slave-gp6n"
server_key: "554374d281f8ff64651b226625a83589"
server_key_iv: "f86c18fa5b77b352977e7e0202aedca6"
server_key_version: "encryptionkey@0"
I20260809 13:02:00.943161 14414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "665204fe600542d5b310db048f57f8ec"
format_stamp: "Formatted at 2026-08-09 13:02:00 on dist-test-slave-gp6n"
server_key: "554374d281f8ff64651b226625a83589"
server_key_iv: "f86c18fa5b77b352977e7e0202aedca6"
server_key_version: "encryptionkey@0"
I20260809 13:02:00.948848 14414 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:02:00.952523 14430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:02:00.953969 14414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:02:00.954173 14414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "665204fe600542d5b310db048f57f8ec"
format_stamp: "Formatted at 2026-08-09 13:02:00 on dist-test-slave-gp6n"
server_key: "554374d281f8ff64651b226625a83589"
server_key_iv: "f86c18fa5b77b352977e7e0202aedca6"
server_key_version: "encryptionkey@0"
I20260809 13:02:00.954396 14414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:02:00.960295 14414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:02:00.964347 14414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:02:00.964658 14414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:02:00.983969 14414 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38525
I20260809 13:02:00.984023 14490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38525 every 8 connection(s)
I20260809 13:02:00.985443 14414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:02:00.987430 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 14414
I20260809 13:02:00.987646 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:02:00.988243 22445 external_mini_cluster.cc:1428] Setting key 7f695ef8abd2d54e4f31084c0f821fa3
I20260809 13:02:00.990597 14491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:02:00.996393 14495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:00.990175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:51489 (local address 127.0.0.1:38525)
0809 13:02:00.991057 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:00.991069 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:00.991111 (+ 42us) server_negotiation.cc:407] Connection header received
0809 13:02:00.992751 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:00.992773 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:00.993146 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:00.993506 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:00.994750 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:00.994757 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:00.994769 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:02:00.994784 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:00.994862 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:02:00.994882 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:00.995077 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:00.995187 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 13:02:00.995401 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":422,"threads_started":1}
I20260809 13:02:01.000614 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:02:01.004240 14491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec: Bootstrap starting.
I20260809 13:02:01.008164 14491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec: Neither blocks nor log segments found. Creating new log.
I20260809 13:02:01.008183 14495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:01.005423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.44:35493 (local address 127.0.0.1:38525)
0809 13:02:01.005595 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:01.005600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:01.006086 (+ 486us) server_negotiation.cc:407] Connection header received
0809 13:02:01.006290 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:01.006302 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:01.006351 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:01.006501 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:01.007471 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:01.007476 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:01.007481 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:02:01.007499 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:01.007566 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:02:01.007582 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:01.007705 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:01.007831 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 13:02:01.007898 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 13:02:01.009758 14491 log.cc:824] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec: Log is configured to *not* fsync() on all Append() calls
I20260809 13:02:01.012744 14491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec: No bootstrap required, opened a new log
I20260809 13:02:01.021677 14491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "665204fe600542d5b310db048f57f8ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38525 } }
I20260809 13:02:01.022084 14491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:02:01.022167 14491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 665204fe600542d5b310db048f57f8ec, State: Initialized, Role: FOLLOWER
I20260809 13:02:01.022864 14491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "665204fe600542d5b310db048f57f8ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38525 } }
I20260809 13:02:01.023084 14491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:02:01.023197 14491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:02:01.023344 14491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:02:01.026177 14491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "665204fe600542d5b310db048f57f8ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38525 } }
I20260809 13:02:01.026779 14491 leader_election.cc:302] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 665204fe600542d5b310db048f57f8ec; no voters:
I20260809 13:02:01.027354 14491 leader_election.cc:288] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:02:01.027544 14503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:02:01.028314 14503 raft_consensus.cc:686] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec [term 1 LEADER]: Becoming Leader. State: Replica: 665204fe600542d5b310db048f57f8ec, State: Running, Role: LEADER
I20260809 13:02:01.028977 14503 consensus_queue.cc:237] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "665204fe600542d5b310db048f57f8ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38525 } }
I20260809 13:02:01.029920 14491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:02:01.032380 14504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "665204fe600542d5b310db048f57f8ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "665204fe600542d5b310db048f57f8ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38525 } } }
I20260809 13:02:01.032842 14504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:01.033610 14505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 665204fe600542d5b310db048f57f8ec. Latest consensus state: current_term: 1 leader_uuid: "665204fe600542d5b310db048f57f8ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "665204fe600542d5b310db048f57f8ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38525 } } }
I20260809 13:02:01.033979 14505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:01.035231 14512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:02:01.041435 14512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:02:01.050980 14512 catalog_manager.cc:1347] Generated new cluster ID: 0a1e7bb7c22f4b589587f7dfec1b630d
I20260809 13:02:01.051124 14512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:02:01.083249 14512 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:02:01.083737 14512 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:02:01.094165 14512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 665204fe600542d5b310db048f57f8ec: Generated new TSK 0
I20260809 13:02:01.095077 14512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:02:01.104941 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 14414
W20260809 13:02:01.106106 14484 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.44:51489
W20260809 13:02:01.106925 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 14414: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:02:01.107082 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 14414
2026-08-09T13:02:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2105 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:02:01 dist-test-slave-gp6n krb5kdc[14527](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:02:01 dist-test-slave-gp6n krb5kdc[14527](info): set up 2 sockets
Aug 09 13:02:01 dist-test-slave-gp6n krb5kdc[14527](info): commencing operation
krb5kdc: starting...
W20260809 13:02:02.330964 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.004s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:02:02 dist-test-slave-gp6n krb5kdc[14527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:02:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:02:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 13:02:02.660033 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:44393
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:33985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:44393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:02:03.005137 14540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:02:03.005465 14540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:02:03.005656 14540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:02:03.013216 14540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:02:03.013326 14540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:02:03.013412 14540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:02:03.013509 14540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:02:03.013612 14540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:02:03.023710 14540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:44393
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:44393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:02:03.025053 14540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:02:03.026441 14540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:02:03.038532 14546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:03.038452 14549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:03.038491 14547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:02:03.039500 14540 server_base.cc:1034] running on GCE node
I20260809 13:02:03.040309 14540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:02:03.041787 14540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:02:03.042984 14540 hybrid_clock.cc:648] HybridClock initialized: now 1786280523042942 us; error 41 us; skew 500 ppm
I20260809 13:02:03.043360 14540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:02:03 dist-test-slave-gp6n krb5kdc[14527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280523, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:02:03.051760 14540 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 13:02:03.053678 14540 webserver.cc:460] Webserver started at http://127.21.235.126:36731/ using document root <none> and password file <none>
I20260809 13:02:03.054790 14540 fs_manager.cc:359] Metadata directory not provided
I20260809 13:02:03.054967 14540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:02:03.055380 14540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:02:03.060065 14540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "62d0fd8fa46846a5985a6769e2a1a5fd"
format_stamp: "Formatted at 2026-08-09 13:02:03 on dist-test-slave-gp6n"
server_key: "7fd03093ceb99e44873151c0b5a82b85"
server_key_iv: "5895527648922cef11f836c7100bd267"
server_key_version: "encryptionkey@0"
I20260809 13:02:03.061299 14540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "62d0fd8fa46846a5985a6769e2a1a5fd"
format_stamp: "Formatted at 2026-08-09 13:02:03 on dist-test-slave-gp6n"
server_key: "7fd03093ceb99e44873151c0b5a82b85"
server_key_iv: "5895527648922cef11f836c7100bd267"
server_key_version: "encryptionkey@0"
I20260809 13:02:03.067248 14540 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 13:02:03.070765 14556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:02:03.072129 14540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 13:02:03.072326 14540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "62d0fd8fa46846a5985a6769e2a1a5fd"
format_stamp: "Formatted at 2026-08-09 13:02:03 on dist-test-slave-gp6n"
server_key: "7fd03093ceb99e44873151c0b5a82b85"
server_key_iv: "5895527648922cef11f836c7100bd267"
server_key_version: "encryptionkey@0"
I20260809 13:02:03.072526 14540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:02:03.095328 14540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:02:03.101521 14540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:02:03.101881 14540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:02:03.124847 14611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:44393 every 8 connection(s)
I20260809 13:02:03.124902 14540 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:44393
I20260809 13:02:03.127046 14540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:02:03.132164 14612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:02:03.133479 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 14540
I20260809 13:02:03.133718 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:02:03.134220 22445 external_mini_cluster.cc:1428] Setting key 55fa1ab9e493b46ead1b7bea9f8201af
I20260809 13:02:03.147409 14612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd: Bootstrap starting.
Aug 09 13:02:03 dist-test-slave-gp6n krb5kdc[14527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:02:03.150924 14612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd: Neither blocks nor log segments found. Creating new log.
I20260809 13:02:03.152381 14612 log.cc:824] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd: Log is configured to *not* fsync() on all Append() calls
I20260809 13:02:03.155921 14612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd: No bootstrap required, opened a new log
I20260809 13:02:03.160043 14617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:03.136399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35903 (local address 127.21.235.126:44393)
0809 13:02:03.137291 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:03.137313 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:03.137377 (+ 64us) server_negotiation.cc:407] Connection header received
0809 13:02:03.139210 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:03.139242 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:03.139792 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:03.140279 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:03.141633 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:03.143721 (+ 2088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:03.145055 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:03.146158 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:03.146340 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:03.150838 (+ 4498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:03.150852 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:03.150882 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:03.150949 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:03.154946 (+ 3997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:03.155832 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:03.155840 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:03.155848 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:03.156113 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:03.156604 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:03.156611 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:03.156617 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:03.157344 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:03.157663 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:03.158126 (+ 463us) server_negotiation.cc:299] Negotiation successful
0809 13:02:03.158508 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":390,"threads_started":1}
I20260809 13:02:03.164108 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:02:03.165709 14612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d0fd8fa46846a5985a6769e2a1a5fd" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 44393 } }
I20260809 13:02:03.166057 14612 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:02:03.166155 14612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d0fd8fa46846a5985a6769e2a1a5fd, State: Initialized, Role: FOLLOWER
I20260809 13:02:03.166870 14612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d0fd8fa46846a5985a6769e2a1a5fd" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 44393 } }
I20260809 13:02:03.167122 14612 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:02:03.167271 14612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:02:03.167448 14612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:02:03.170559 14612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d0fd8fa46846a5985a6769e2a1a5fd" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 44393 } }
I20260809 13:02:03.171196 14612 leader_election.cc:302] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62d0fd8fa46846a5985a6769e2a1a5fd; no voters:
I20260809 13:02:03.171994 14612 leader_election.cc:288] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:02:03.172111 14625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:02:03.174646 14612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:02:03.178913 14625 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd [term 1 LEADER]: Becoming Leader. State: Replica: 62d0fd8fa46846a5985a6769e2a1a5fd, State: Running, Role: LEADER
I20260809 13:02:03.179790 14625 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d0fd8fa46846a5985a6769e2a1a5fd" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 44393 } }
I20260809 13:02:03.183215 14626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62d0fd8fa46846a5985a6769e2a1a5fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d0fd8fa46846a5985a6769e2a1a5fd" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 44393 } } }
I20260809 13:02:03.183130 14627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62d0fd8fa46846a5985a6769e2a1a5fd. Latest consensus state: current_term: 1 leader_uuid: "62d0fd8fa46846a5985a6769e2a1a5fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d0fd8fa46846a5985a6769e2a1a5fd" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 44393 } } }
I20260809 13:02:03.183569 14627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:03.186300 14626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:03.187546 14617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:03.168607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59087 (local address 127.21.235.126:44393)
0809 13:02:03.168837 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:03.168846 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:03.169213 (+ 367us) server_negotiation.cc:407] Connection header received
0809 13:02:03.169398 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:03.169403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:03.169464 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:03.169544 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:03.171565 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:03.172615 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:03.174104 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:03.174632 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:03.174773 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:03.177770 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:03.177775 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:03.177778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:03.177807 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:03.181901 (+ 4094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:03.183251 (+ 1350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:03.183263 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:03.183269 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:03.183471 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:03.184195 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:03.184203 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:03.184208 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:03.184726 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:03.185085 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:03.186757 (+ 1672us) server_negotiation.cc:299] Negotiation successful
0809 13:02:03.187014 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 13:02:03.191401 14636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:02:03.196627 14636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:02:03.205605 14636 catalog_manager.cc:1347] Generated new cluster ID: c66d8412713d4166a757bc7e1b91a346
I20260809 13:02:03.205730 14636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:02:03.230466 14636 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:02:03.232443 14636 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:02:03.239964 14636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62d0fd8fa46846a5985a6769e2a1a5fd: Generated new TSK 0
I20260809 13:02:03.241083 14636 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:02:03.249341 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 14540
I20260809 13:02:03.250608 14606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:02:03.342953 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 14540
W20260809 13:02:03.364441 14525 connection.cc:535] client connection to 127.21.235.126:44393 recv error: Network error: failed to read from TLS socket (remote: 127.21.235.126:44393): Connection reset by peer (error 104)
2026-08-09T13:02:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2235 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:02:03 dist-test-slave-gp6n krb5kdc[14650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:02:03 dist-test-slave-gp6n krb5kdc[14650](info): set up 2 sockets
Aug 09 13:02:03 dist-test-slave-gp6n krb5kdc[14650](info): commencing operation
krb5kdc: starting...
W20260809 13:02:04.553565 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:02:04 dist-test-slave-gp6n krb5kdc[14650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:02:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:02:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 13:02:04.890601 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:39473
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:37673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:39473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:02:05.247707 14664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:02:05.248019 14664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:02:05.248199 14664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:02:05.255563 14664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:02:05.255678 14664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:02:05.255789 14664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:02:05.255903 14664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:02:05.255990 14664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:02:05.266002 14664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:37673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:39473
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:39473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:02:05.267374 14664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:02:05.268754 14664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:02:05.281124 14671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:05.281162 14674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:05.281183 14672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:02:05.281847 14664 server_base.cc:1034] running on GCE node
I20260809 13:02:05.282790 14664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:02:05.284317 14664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:02:05.285573 14664 hybrid_clock.cc:648] HybridClock initialized: now 1786280525285521 us; error 45 us; skew 500 ppm
I20260809 13:02:05.285959 14664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:02:05 dist-test-slave-gp6n krb5kdc[14650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280525, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:02:05.289341 14664 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 13:02:05.290947 14664 webserver.cc:460] Webserver started at http://127.21.235.126:43287/ using document root <none> and password file <none>
I20260809 13:02:05.291585 14664 fs_manager.cc:359] Metadata directory not provided
I20260809 13:02:05.291697 14664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:02:05.291965 14664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:02:05.295054 14664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "54cc002b464f4e228217dc49b9c1a6de"
format_stamp: "Formatted at 2026-08-09 13:02:05 on dist-test-slave-gp6n"
server_key: "48da7b1ee018207855d23bf3eeb27d1e"
server_key_iv: "1b4f6e37fca5ee8d5dddc6c804dc8795"
server_key_version: "encryptionkey@0"
I20260809 13:02:05.295771 14664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "54cc002b464f4e228217dc49b9c1a6de"
format_stamp: "Formatted at 2026-08-09 13:02:05 on dist-test-slave-gp6n"
server_key: "48da7b1ee018207855d23bf3eeb27d1e"
server_key_iv: "1b4f6e37fca5ee8d5dddc6c804dc8795"
server_key_version: "encryptionkey@0"
I20260809 13:02:05.300974 14664 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:02:05.304452 14684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:02:05.305825 14664 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:02:05.306027 14664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "54cc002b464f4e228217dc49b9c1a6de"
format_stamp: "Formatted at 2026-08-09 13:02:05 on dist-test-slave-gp6n"
server_key: "48da7b1ee018207855d23bf3eeb27d1e"
server_key_iv: "1b4f6e37fca5ee8d5dddc6c804dc8795"
server_key_version: "encryptionkey@0"
I20260809 13:02:05.306200 14664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:02:05.326206 14664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:02:05.330961 14664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:02:05.331276 14664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:02:05.350660 14664 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:39473
I20260809 13:02:05.350693 14738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:39473 every 8 connection(s)
I20260809 13:02:05.352136 14664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:02:05.352918 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 14664
I20260809 13:02:05.353127 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:02:05.353724 22445 external_mini_cluster.cc:1428] Setting key 62f05134ca320a527ff811d9c4985734
I20260809 13:02:05.356709 14739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:02:05.367686 14742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:05.355633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43555 (local address 127.21.235.126:39473)
0809 13:02:05.356470 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:05.356482 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:05.356524 (+ 42us) server_negotiation.cc:407] Connection header received
0809 13:02:05.358198 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:05.358219 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:05.358676 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:05.359000 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:05.359610 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:05.361010 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:05.362922 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:05.363618 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:05.363743 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:05.365095 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:05.365103 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:05.365116 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:02:05.365714 (+ 598us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:05.365791 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:02:05.365824 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:05.365950 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:05.366427 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 13:02:05.366648 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":381,"threads_started":1}
I20260809 13:02:05.369165 14739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de: Bootstrap starting.
I20260809 13:02:05.371965 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:02:05.372844 14739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de: Neither blocks nor log segments found. Creating new log.
I20260809 13:02:05.373847 14739 log.cc:824] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de: Log is configured to *not* fsync() on all Append() calls
I20260809 13:02:05.377346 14739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de: No bootstrap required, opened a new log
I20260809 13:02:05.382938 14742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:05.376612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32875 (local address 127.21.235.126:39473)
0809 13:02:05.376801 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:05.376809 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:05.377288 (+ 479us) server_negotiation.cc:407] Connection header received
0809 13:02:05.377516 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:05.377520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:05.377579 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:05.377691 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:05.378193 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:05.379051 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:05.380382 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:05.380845 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:05.380977 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:05.382175 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:05.382180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:05.382183 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:02:05.382200 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:05.382238 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:02:05.382256 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:05.382450 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:05.382627 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 13:02:05.382684 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 13:02:05.386050 14739 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54cc002b464f4e228217dc49b9c1a6de" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 39473 } }
I20260809 13:02:05.386571 14739 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:02:05.386694 14739 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54cc002b464f4e228217dc49b9c1a6de, State: Initialized, Role: FOLLOWER
I20260809 13:02:05.387388 14739 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54cc002b464f4e228217dc49b9c1a6de" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 39473 } }
I20260809 13:02:05.387645 14739 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:02:05.387796 14739 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:02:05.387979 14739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:02:05.390434 14739 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54cc002b464f4e228217dc49b9c1a6de" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 39473 } }
I20260809 13:02:05.391048 14739 leader_election.cc:302] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54cc002b464f4e228217dc49b9c1a6de; no voters:
I20260809 13:02:05.391809 14739 leader_election.cc:288] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:02:05.391933 14749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:02:05.392632 14749 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de [term 1 LEADER]: Becoming Leader. State: Replica: 54cc002b464f4e228217dc49b9c1a6de, State: Running, Role: LEADER
I20260809 13:02:05.393297 14749 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54cc002b464f4e228217dc49b9c1a6de" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 39473 } }
I20260809 13:02:05.394220 14739 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:02:05.396526 14750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54cc002b464f4e228217dc49b9c1a6de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54cc002b464f4e228217dc49b9c1a6de" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 39473 } } }
I20260809 13:02:05.396971 14750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:05.396653 14751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54cc002b464f4e228217dc49b9c1a6de. Latest consensus state: current_term: 1 leader_uuid: "54cc002b464f4e228217dc49b9c1a6de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54cc002b464f4e228217dc49b9c1a6de" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 39473 } } }
I20260809 13:02:05.397089 14751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:05.399340 14759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:02:05.405344 14759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:02:05.414638 14759 catalog_manager.cc:1347] Generated new cluster ID: afd5164e28344733b4acf7166b7edaae
I20260809 13:02:05.414760 14759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:02:05.425724 14759 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:02:05.427597 14759 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:02:05.435835 14759 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54cc002b464f4e228217dc49b9c1a6de: Generated new TSK 0
I20260809 13:02:05.436733 14759 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:02:05.448426 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 14664
W20260809 13:02:05.449656 14733 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43555
W20260809 13:02:05.450519 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 14664: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:02:05.450697 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 14664
W20260809 13:02:05.474972 14647 connection.cc:535] client connection to 127.21.235.126:39473 recv error: Network error: failed to read from TLS socket (remote: 127.21.235.126:39473): Connection reset by peer (error 104)
2026-08-09T13:02:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2110 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:02:05 dist-test-slave-gp6n krb5kdc[14774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:02:05 dist-test-slave-gp6n krb5kdc[14774](info): set up 2 sockets
Aug 09 13:02:05 dist-test-slave-gp6n krb5kdc[14774](info): commencing operation
krb5kdc: starting...
W20260809 13:02:06.645402 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.004s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:02:06 dist-test-slave-gp6n krb5kdc[14774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:02:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:02:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 13:02:06.974681 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:37163
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:37163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:02:07.327661 14789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:02:07.327976 14789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:02:07.328173 14789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:02:07.335668 14789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:02:07.335790 14789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:02:07.335868 14789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:02:07.335970 14789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:02:07.336063 14789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:02:07.346276 14789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:37163
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:37163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:02:07.347728 14789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:02:07.349123 14789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:02:07.361394 14797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:07.361482 14795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:07.361716 14794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:02:07.362347 14789 server_base.cc:1034] running on GCE node
I20260809 13:02:07.363163 14789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:02:07.364727 14789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:02:07.365990 14789 hybrid_clock.cc:648] HybridClock initialized: now 1786280527365924 us; error 65 us; skew 500 ppm
I20260809 13:02:07.366392 14789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:02:07 dist-test-slave-gp6n krb5kdc[14774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280527, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:02:07.369714 14789 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 13:02:07.371343 14789 webserver.cc:460] Webserver started at http://127.21.235.126:43031/ using document root <none> and password file <none>
I20260809 13:02:07.372062 14789 fs_manager.cc:359] Metadata directory not provided
I20260809 13:02:07.372217 14789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:02:07.372541 14789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:02:07.375696 14789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "e63d817e15054c27be3e67404fcac103"
format_stamp: "Formatted at 2026-08-09 13:02:07 on dist-test-slave-gp6n"
server_key: "14f141676e55bae2e80f589fd2a8ccf0"
server_key_iv: "5d9d4100a48df4f5b8a1083c73c8dc0b"
server_key_version: "encryptionkey@0"
I20260809 13:02:07.376509 14789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "e63d817e15054c27be3e67404fcac103"
format_stamp: "Formatted at 2026-08-09 13:02:07 on dist-test-slave-gp6n"
server_key: "14f141676e55bae2e80f589fd2a8ccf0"
server_key_iv: "5d9d4100a48df4f5b8a1083c73c8dc0b"
server_key_version: "encryptionkey@0"
I20260809 13:02:07.381839 14789 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260809 13:02:07.385481 14804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:02:07.386945 14789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 13:02:07.387164 14789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "e63d817e15054c27be3e67404fcac103"
format_stamp: "Formatted at 2026-08-09 13:02:07 on dist-test-slave-gp6n"
server_key: "14f141676e55bae2e80f589fd2a8ccf0"
server_key_iv: "5d9d4100a48df4f5b8a1083c73c8dc0b"
server_key_version: "encryptionkey@0"
I20260809 13:02:07.387421 14789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:02:07.393963 14789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:02:07.398340 14789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:02:07.398681 14789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:02:07.417981 14789 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:37163
I20260809 13:02:07.418067 14862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:37163 every 8 connection(s)
I20260809 13:02:07.419432 14789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:02:07.423561 14863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:02:07.425170 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 14789
I20260809 13:02:07.425379 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:02:07.426003 22445 external_mini_cluster.cc:1428] Setting key 3edb6b4d447f90c8c22572b5f882e6da
I20260809 13:02:07.434245 14866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:07.428116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55233 (local address 127.21.235.126:37163)
0809 13:02:07.429064 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:07.429076 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:07.429118 (+ 42us) server_negotiation.cc:407] Connection header received
0809 13:02:07.430832 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:07.430855 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:07.431196 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:07.431606 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:07.432680 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:07.432686 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:07.432701 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:02:07.432715 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:07.432780 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:02:07.432799 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:07.433039 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:07.433181 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 13:02:07.433361 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":436,"threads_started":1}
I20260809 13:02:07.438205 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:02:07.438905 14863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103: Bootstrap starting.
I20260809 13:02:07.442188 14863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103: Neither blocks nor log segments found. Creating new log.
I20260809 13:02:07.443756 14863 log.cc:824] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103: Log is configured to *not* fsync() on all Append() calls
I20260809 13:02:07.445645 14866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:07.442764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35087 (local address 127.21.235.126:37163)
0809 13:02:07.442934 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:07.442938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:07.443469 (+ 531us) server_negotiation.cc:407] Connection header received
0809 13:02:07.443650 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:07.443656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:07.443715 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:07.443823 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:07.444910 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:07.444916 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:07.444920 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:02:07.444940 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:07.444988 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:02:07.445008 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:07.445112 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:07.445292 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 13:02:07.445336 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 13:02:07.447213 14863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103: No bootstrap required, opened a new log
I20260809 13:02:07.455670 14863 raft_consensus.cc:348] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63d817e15054c27be3e67404fcac103" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 37163 } }
I20260809 13:02:07.456110 14863 raft_consensus.cc:374] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:02:07.456228 14863 raft_consensus.cc:729] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63d817e15054c27be3e67404fcac103, State: Initialized, Role: FOLLOWER
I20260809 13:02:07.457007 14863 consensus_queue.cc:260] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63d817e15054c27be3e67404fcac103" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 37163 } }
I20260809 13:02:07.457333 14863 raft_consensus.cc:388] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:02:07.457517 14863 raft_consensus.cc:482] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:02:07.457705 14863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:02:07.460350 14863 raft_consensus.cc:504] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63d817e15054c27be3e67404fcac103" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 37163 } }
I20260809 13:02:07.460924 14863 leader_election.cc:302] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e63d817e15054c27be3e67404fcac103; no voters:
I20260809 13:02:07.461601 14863 leader_election.cc:288] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:02:07.461740 14875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:02:07.463078 14875 raft_consensus.cc:686] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103 [term 1 LEADER]: Becoming Leader. State: Replica: e63d817e15054c27be3e67404fcac103, State: Running, Role: LEADER
I20260809 13:02:07.463629 14875 consensus_queue.cc:237] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63d817e15054c27be3e67404fcac103" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 37163 } }
I20260809 13:02:07.464210 14863 sys_catalog.cc:564] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:02:07.466878 14876 sys_catalog.cc:455] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e63d817e15054c27be3e67404fcac103" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63d817e15054c27be3e67404fcac103" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 37163 } } }
I20260809 13:02:07.467478 14876 sys_catalog.cc:458] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103 [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:07.467012 14877 sys_catalog.cc:455] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e63d817e15054c27be3e67404fcac103. Latest consensus state: current_term: 1 leader_uuid: "e63d817e15054c27be3e67404fcac103" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63d817e15054c27be3e67404fcac103" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 37163 } } }
I20260809 13:02:07.471113 14877 sys_catalog.cc:458] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103 [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:07.472204 14885 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:02:07.478258 14885 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:02:07.488296 14885 catalog_manager.cc:1347] Generated new cluster ID: 431b229e6f48488a869e211bb028e2b1
I20260809 13:02:07.488436 14885 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:02:07.504485 14885 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:02:07.504968 14885 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:02:07.520768 14885 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e63d817e15054c27be3e67404fcac103: Generated new TSK 0
I20260809 13:02:07.522001 14885 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:02:07.535400 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 14789
W20260809 13:02:07.536821 14857 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55233
W20260809 13:02:07.537555 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 14789: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:02:07.537696 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 14789
2026-08-09T13:02:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2084 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:02:07 dist-test-slave-gp6n krb5kdc[14899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:02:07 dist-test-slave-gp6n krb5kdc[14899](info): set up 2 sockets
Aug 09 13:02:07 dist-test-slave-gp6n krb5kdc[14899](info): commencing operation
krb5kdc: starting...
W20260809 13:02:08.795964 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:02:09 dist-test-slave-gp6n krb5kdc[14899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:02:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:02:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 13:02:09.135872 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:34883
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:34407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:34883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:02:09.487262 14914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:02:09.487564 14914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:02:09.487748 14914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:02:09.495132 14914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:02:09.495255 14914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:02:09.495383 14914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:02:09.495487 14914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:02:09.495572 14914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:02:09.505694 14914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:34407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:34883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:34883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:02:09.507166 14914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:02:09.508591 14914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:02:09.521353 14921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:09.521448 14920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:09.521448 14923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:02:09.522883 14914 server_base.cc:1034] running on GCE node
I20260809 13:02:09.523690 14914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:02:09.525156 14914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:02:09.526376 14914 hybrid_clock.cc:648] HybridClock initialized: now 1786280529526312 us; error 55 us; skew 500 ppm
I20260809 13:02:09.526763 14914 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:02:09 dist-test-slave-gp6n krb5kdc[14899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280529, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:02:09.535385 14914 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 13:02:09.537106 14914 webserver.cc:460] Webserver started at http://127.21.235.126:46807/ using document root <none> and password file <none>
I20260809 13:02:09.537882 14914 fs_manager.cc:359] Metadata directory not provided
I20260809 13:02:09.538033 14914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:02:09.538355 14914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:02:09.541515 14914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "e7f38f40c03c40a98485a9bad5c73218"
format_stamp: "Formatted at 2026-08-09 13:02:09 on dist-test-slave-gp6n"
server_key: "b6e97b1430ba61fb2d5f0861e8f5ece3"
server_key_iv: "f0e21c8f582ce90515ab4718a8ca36e6"
server_key_version: "encryptionkey@0"
I20260809 13:02:09.542322 14914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "e7f38f40c03c40a98485a9bad5c73218"
format_stamp: "Formatted at 2026-08-09 13:02:09 on dist-test-slave-gp6n"
server_key: "b6e97b1430ba61fb2d5f0861e8f5ece3"
server_key_iv: "f0e21c8f582ce90515ab4718a8ca36e6"
server_key_version: "encryptionkey@0"
I20260809 13:02:09.547788 14914 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 13:02:09.551337 14933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:02:09.552738 14914 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 13:02:09.552938 14914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "e7f38f40c03c40a98485a9bad5c73218"
format_stamp: "Formatted at 2026-08-09 13:02:09 on dist-test-slave-gp6n"
server_key: "b6e97b1430ba61fb2d5f0861e8f5ece3"
server_key_iv: "f0e21c8f582ce90515ab4718a8ca36e6"
server_key_version: "encryptionkey@0"
I20260809 13:02:09.553153 14914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:02:09.567669 14914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:02:09.572227 14914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:02:09.572542 14914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:02:09.591534 14914 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:34883
I20260809 13:02:09.591566 14988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:34883 every 8 connection(s)
I20260809 13:02:09.592881 14914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:02:09.596556 14989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:02:09.598793 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 14914
I20260809 13:02:09.598978 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:02:09.599529 22445 external_mini_cluster.cc:1428] Setting key 9cc3513e1a904bd10775224bc2dfc6c9
I20260809 13:02:09.609071 14989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218: Bootstrap starting.
I20260809 13:02:09.613011 14989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218: Neither blocks nor log segments found. Creating new log.
Aug 09 13:02:09 dist-test-slave-gp6n krb5kdc[14899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:02:09.614221 14989 log.cc:824] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218: Log is configured to *not* fsync() on all Append() calls
I20260809 13:02:09.617713 14989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218: No bootstrap required, opened a new log
I20260809 13:02:09.623302 14993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:09.601306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52097 (local address 127.21.235.126:34883)
0809 13:02:09.602138 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:09.602149 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:09.602195 (+ 46us) server_negotiation.cc:407] Connection header received
0809 13:02:09.603958 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:09.603980 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:09.604370 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:09.604677 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:09.606539 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:09.607880 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:09.609641 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:09.610271 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:09.610471 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:09.615422 (+ 4951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:09.615436 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:09.615457 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:09.615505 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:09.618903 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:09.619817 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:09.619824 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:09.619829 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:09.620012 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:09.620461 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:09.620464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:09.620466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:09.620991 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:09.621213 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:09.621708 (+ 495us) server_negotiation.cc:299] Negotiation successful
0809 13:02:09.622113 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":397,"threads_started":1}
I20260809 13:02:09.627497 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:02:09.628324 14989 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7f38f40c03c40a98485a9bad5c73218" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34883 } }
I20260809 13:02:09.628698 14989 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:02:09.628827 14989 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7f38f40c03c40a98485a9bad5c73218, State: Initialized, Role: FOLLOWER
I20260809 13:02:09.629487 14989 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7f38f40c03c40a98485a9bad5c73218" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34883 } }
I20260809 13:02:09.629768 14989 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:02:09.629930 14989 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:02:09.630124 14989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:02:09.633148 14989 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7f38f40c03c40a98485a9bad5c73218" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34883 } }
I20260809 13:02:09.633677 14989 leader_election.cc:302] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7f38f40c03c40a98485a9bad5c73218; no voters:
I20260809 13:02:09.634522 14989 leader_election.cc:288] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:02:09.634641 15002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:02:09.637334 14989 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:02:09.638697 15002 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218 [term 1 LEADER]: Becoming Leader. State: Replica: e7f38f40c03c40a98485a9bad5c73218, State: Running, Role: LEADER
I20260809 13:02:09.639446 15002 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7f38f40c03c40a98485a9bad5c73218" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34883 } }
I20260809 13:02:09.643842 15004 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7f38f40c03c40a98485a9bad5c73218. Latest consensus state: current_term: 1 leader_uuid: "e7f38f40c03c40a98485a9bad5c73218" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7f38f40c03c40a98485a9bad5c73218" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34883 } } }
I20260809 13:02:09.644413 15004 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218 [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:09.643790 15003 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7f38f40c03c40a98485a9bad5c73218" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7f38f40c03c40a98485a9bad5c73218" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34883 } } }
I20260809 13:02:09.646620 15003 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218 [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:09.647775 15012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:02:09.650993 14993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:09.632021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46659 (local address 127.21.235.126:34883)
0809 13:02:09.632207 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:09.632214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:09.632661 (+ 447us) server_negotiation.cc:407] Connection header received
0809 13:02:09.632856 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:09.632861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:09.632940 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:09.633040 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:09.634733 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:09.635988 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:09.637265 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:09.637795 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:09.637930 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:09.640857 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:09.640866 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:09.640872 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:09.640914 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:09.646211 (+ 5297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:09.647268 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:09.647274 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:09.647280 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:09.647447 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:09.647979 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:09.647984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:09.647992 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:09.648527 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:09.650083 (+ 1556us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:09.650310 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 13:02:09.650579 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 13:02:09.653563 15012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:02:09.662612 15012 catalog_manager.cc:1347] Generated new cluster ID: 2f08e9cb42e941f4a0acc28b7dc059ba
I20260809 13:02:09.662730 15012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:02:09.682763 15012 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:02:09.684248 15012 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:02:09.689409 15012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7f38f40c03c40a98485a9bad5c73218: Generated new TSK 0
I20260809 13:02:09.690271 15012 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:02:09.705543 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 14914
I20260809 13:02:09.706796 14982 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:02:09.799393 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 14914
2026-08-09T13:02:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2260 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:02:09 dist-test-slave-gp6n krb5kdc[15025](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:02:09 dist-test-slave-gp6n krb5kdc[15025](info): set up 2 sockets
Aug 09 13:02:09 dist-test-slave-gp6n krb5kdc[15025](info): commencing operation
krb5kdc: starting...
W20260809 13:02:11.005690 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:02:11 dist-test-slave-gp6n krb5kdc[15025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:02:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:02:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 13:02:11.337841 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:34465
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:41559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:34465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:02:11.687260 15040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:02:11.687597 15040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:02:11.687767 15040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:02:11.695129 15040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:02:11.695238 15040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:02:11.695355 15040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:02:11.695458 15040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:02:11.695544 15040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:02:11.705350 15040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:41559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:34465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:34465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:02:11.706699 15040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:02:11.708061 15040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:02:11.719985 15045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:11.720037 15048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:11.720067 15046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:02:11.720935 15040 server_base.cc:1034] running on GCE node
I20260809 13:02:11.721757 15040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:02:11.723306 15040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:02:11.724550 15040 hybrid_clock.cc:648] HybridClock initialized: now 1786280531724515 us; error 58 us; skew 500 ppm
I20260809 13:02:11.724910 15040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:02:11 dist-test-slave-gp6n krb5kdc[15025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280531, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:02:11.728390 15040 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 13:02:11.730103 15040 webserver.cc:460] Webserver started at http://127.21.235.126:39329/ using document root <none> and password file <none>
I20260809 13:02:11.730940 15040 fs_manager.cc:359] Metadata directory not provided
I20260809 13:02:11.731106 15040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:02:11.731453 15040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:02:11.734830 15040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "fe88410e8fb34772adb5a5ef282a01b6"
format_stamp: "Formatted at 2026-08-09 13:02:11 on dist-test-slave-gp6n"
server_key: "19b0c5500e0e4e007ff5f8651a742464"
server_key_iv: "3378b85f04c5c4a41b4f404df7e108e7"
server_key_version: "encryptionkey@0"
I20260809 13:02:11.735682 15040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "fe88410e8fb34772adb5a5ef282a01b6"
format_stamp: "Formatted at 2026-08-09 13:02:11 on dist-test-slave-gp6n"
server_key: "19b0c5500e0e4e007ff5f8651a742464"
server_key_iv: "3378b85f04c5c4a41b4f404df7e108e7"
server_key_version: "encryptionkey@0"
I20260809 13:02:11.741528 15040 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 13:02:11.745551 15057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:02:11.747049 15040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:02:11.747248 15040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "fe88410e8fb34772adb5a5ef282a01b6"
format_stamp: "Formatted at 2026-08-09 13:02:11 on dist-test-slave-gp6n"
server_key: "19b0c5500e0e4e007ff5f8651a742464"
server_key_iv: "3378b85f04c5c4a41b4f404df7e108e7"
server_key_version: "encryptionkey@0"
I20260809 13:02:11.747473 15040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:02:11.776786 15040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:02:11.781530 15040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:02:11.781847 15040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:02:11.801445 15040 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:34465
I20260809 13:02:11.801478 15114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:34465 every 8 connection(s)
I20260809 13:02:11.802908 15040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:02:11.806872 15115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:02:11.809700 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 15040
I20260809 13:02:11.809883 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:02:11.810469 22445 external_mini_cluster.cc:1428] Setting key 339aef7a2424642a55dfd24f305e0e4e
I20260809 13:02:11.819433 15115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6: Bootstrap starting.
I20260809 13:02:11.823307 15115 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6: Neither blocks nor log segments found. Creating new log.
I20260809 13:02:11.824643 15115 log.cc:824] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6: Log is configured to *not* fsync() on all Append() calls
I20260809 13:02:11.824843 15118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:11.812475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49301 (local address 127.21.235.126:34465)
0809 13:02:11.813380 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:11.813392 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:11.813434 (+ 42us) server_negotiation.cc:407] Connection header received
0809 13:02:11.815205 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:11.815229 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:11.815670 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:11.816056 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:11.816622 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:11.818019 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:11.819911 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:11.820607 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:11.820745 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:11.822102 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:11.822110 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:11.822124 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:02:11.822740 (+ 616us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:11.822819 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:02:11.822852 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:11.822941 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:11.823209 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 13:02:11.823525 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":453,"threads_started":1}
I20260809 13:02:11.827994 15115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6: No bootstrap required, opened a new log
I20260809 13:02:11.828985 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:02:11.838294 15115 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe88410e8fb34772adb5a5ef282a01b6" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34465 } }
I20260809 13:02:11.838855 15115 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:02:11.838986 15115 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe88410e8fb34772adb5a5ef282a01b6, State: Initialized, Role: FOLLOWER
I20260809 13:02:11.839644 15115 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe88410e8fb34772adb5a5ef282a01b6" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34465 } }
I20260809 13:02:11.839918 15115 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:02:11.840066 15115 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:02:11.840279 15115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:02:11.841341 15118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:11.834181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49399 (local address 127.21.235.126:34465)
0809 13:02:11.834465 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:11.834472 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:11.834944 (+ 472us) server_negotiation.cc:407] Connection header received
0809 13:02:11.835100 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:11.835105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:11.835182 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:11.835300 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:11.835784 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:11.836965 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:11.838525 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:11.839171 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:11.839310 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:11.840571 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:11.840578 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:11.840584 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:02:11.840608 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:11.840648 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:02:11.840664 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:11.840765 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:11.840926 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 13:02:11.841026 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 13:02:11.843430 15115 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe88410e8fb34772adb5a5ef282a01b6" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34465 } }
I20260809 13:02:11.843981 15115 leader_election.cc:302] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe88410e8fb34772adb5a5ef282a01b6; no voters:
I20260809 13:02:11.844564 15115 leader_election.cc:288] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:02:11.844707 15125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:02:11.845494 15125 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6 [term 1 LEADER]: Becoming Leader. State: Replica: fe88410e8fb34772adb5a5ef282a01b6, State: Running, Role: LEADER
I20260809 13:02:11.846041 15125 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe88410e8fb34772adb5a5ef282a01b6" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34465 } }
I20260809 13:02:11.846935 15115 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:02:11.849296 15128 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe88410e8fb34772adb5a5ef282a01b6. Latest consensus state: current_term: 1 leader_uuid: "fe88410e8fb34772adb5a5ef282a01b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe88410e8fb34772adb5a5ef282a01b6" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34465 } } }
I20260809 13:02:11.849809 15128 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6 [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:11.854638 15126 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe88410e8fb34772adb5a5ef282a01b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe88410e8fb34772adb5a5ef282a01b6" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 34465 } } }
I20260809 13:02:11.854930 15126 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6 [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:11.855496 15132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:02:11.861096 15132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:02:11.870136 15132 catalog_manager.cc:1347] Generated new cluster ID: 205fc956a94742a3bbab0b9c2f0d24bc
I20260809 13:02:11.870244 15132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:02:11.918560 15132 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:02:11.919888 15132 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:02:11.926731 15132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe88410e8fb34772adb5a5ef282a01b6: Generated new TSK 0
I20260809 13:02:11.927949 15132 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:02:11.948290 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 15040
W20260809 13:02:11.949450 15110 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49301
W20260809 13:02:11.950279 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 15040: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:02:11.950507 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 15040
2026-08-09T13:02:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2151 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:02:12 dist-test-slave-gp6n krb5kdc[15149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:02:12 dist-test-slave-gp6n krb5kdc[15149](info): set up 2 sockets
Aug 09 13:02:12 dist-test-slave-gp6n krb5kdc[15149](info): commencing operation
krb5kdc: starting...
W20260809 13:02:13.169989 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:02:13 dist-test-slave-gp6n krb5kdc[15149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:02:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:02:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 13:02:13.514173 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:41053
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:32989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:41053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:02:13.868738 15163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:02:13.869033 15163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:02:13.869200 15163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:02:13.876766 15163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:02:13.876878 15163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:02:13.876984 15163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:02:13.877071 15163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:02:13.877141 15163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:02:13.887235 15163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:32989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:41053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:41053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:02:13.888625 15163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:02:13.890072 15163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:02:13.902385 15168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:13.902395 15172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:13.902458 15169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:02:13.903460 15163 server_base.cc:1034] running on GCE node
I20260809 13:02:13.904335 15163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:02:13.905829 15163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:02:13.907058 15163 hybrid_clock.cc:648] HybridClock initialized: now 1786280533907016 us; error 30 us; skew 500 ppm
I20260809 13:02:13.907431 15163 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:02:13 dist-test-slave-gp6n krb5kdc[15149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280533, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:02:13.910876 15163 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 13:02:13.912489 15163 webserver.cc:460] Webserver started at http://127.21.235.126:39849/ using document root <none> and password file <none>
I20260809 13:02:13.913237 15163 fs_manager.cc:359] Metadata directory not provided
I20260809 13:02:13.913400 15163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:02:13.913725 15163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:02:13.916951 15163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "f0073c7b2bb64923b98689c6a46ccbea"
format_stamp: "Formatted at 2026-08-09 13:02:13 on dist-test-slave-gp6n"
server_key: "16a658c50251c830d35878352a63c093"
server_key_iv: "5ae52386e42512eb42260bf10b1caa85"
server_key_version: "encryptionkey@0"
I20260809 13:02:13.917814 15163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "f0073c7b2bb64923b98689c6a46ccbea"
format_stamp: "Formatted at 2026-08-09 13:02:13 on dist-test-slave-gp6n"
server_key: "16a658c50251c830d35878352a63c093"
server_key_iv: "5ae52386e42512eb42260bf10b1caa85"
server_key_version: "encryptionkey@0"
I20260809 13:02:13.923491 15163 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 13:02:13.927263 15179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:02:13.928648 15163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:02:13.928850 15163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "f0073c7b2bb64923b98689c6a46ccbea"
format_stamp: "Formatted at 2026-08-09 13:02:13 on dist-test-slave-gp6n"
server_key: "16a658c50251c830d35878352a63c093"
server_key_iv: "5ae52386e42512eb42260bf10b1caa85"
server_key_version: "encryptionkey@0"
I20260809 13:02:13.929078 15163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:02:13.935213 15163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:02:13.939944 15163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:02:13.940253 15163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:02:13.959138 15163 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:41053
I20260809 13:02:13.959169 15236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:41053 every 8 connection(s)
I20260809 13:02:13.960604 15163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:02:13.964581 15237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:02:13.968267 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 15163
I20260809 13:02:13.968461 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:02:13.968897 22445 external_mini_cluster.cc:1428] Setting key 3c8c72ef287be21af972521f0049eab9
I20260809 13:02:13.976950 15241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:13.970970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57713 (local address 127.21.235.126:41053)
0809 13:02:13.971773 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:13.971785 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:13.971833 (+ 48us) server_negotiation.cc:407] Connection header received
0809 13:02:13.973461 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:13.973484 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:13.973854 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:13.974197 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:13.975376 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:13.975383 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:13.975395 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:02:13.975415 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:13.975476 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:02:13.975495 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:13.975668 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:13.975805 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 13:02:13.976020 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":342,"threads_started":1}
I20260809 13:02:13.977779 15237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea: Bootstrap starting.
I20260809 13:02:13.980762 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:02:13.981354 15237 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea: Neither blocks nor log segments found. Creating new log.
I20260809 13:02:13.982443 15237 log.cc:824] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea: Log is configured to *not* fsync() on all Append() calls
I20260809 13:02:13.985603 15237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea: No bootstrap required, opened a new log
I20260809 13:02:13.987825 15241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:13.985180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40861 (local address 127.21.235.126:41053)
0809 13:02:13.985356 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:13.985362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:13.985883 (+ 521us) server_negotiation.cc:407] Connection header received
0809 13:02:13.986014 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:13.986020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:13.986096 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:13.986213 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:13.987166 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:13.987171 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:13.987175 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:02:13.987196 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:13.987241 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:02:13.987260 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:13.987393 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:13.987444 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 13:02:13.987508 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 13:02:13.993796 15237 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0073c7b2bb64923b98689c6a46ccbea" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 41053 } }
I20260809 13:02:13.994212 15237 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:02:13.994343 15237 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0073c7b2bb64923b98689c6a46ccbea, State: Initialized, Role: FOLLOWER
I20260809 13:02:13.995052 15237 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0073c7b2bb64923b98689c6a46ccbea" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 41053 } }
I20260809 13:02:13.995285 15237 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:02:13.995445 15237 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:02:13.995640 15237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:02:13.998358 15237 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0073c7b2bb64923b98689c6a46ccbea" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 41053 } }
I20260809 13:02:13.998966 15237 leader_election.cc:302] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0073c7b2bb64923b98689c6a46ccbea; no voters:
I20260809 13:02:13.999584 15237 leader_election.cc:288] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:02:13.999781 15250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:02:14.000505 15250 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea [term 1 LEADER]: Becoming Leader. State: Replica: f0073c7b2bb64923b98689c6a46ccbea, State: Running, Role: LEADER
I20260809 13:02:14.001108 15250 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0073c7b2bb64923b98689c6a46ccbea" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 41053 } }
I20260809 13:02:14.001914 15237 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:02:14.004478 15252 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0073c7b2bb64923b98689c6a46ccbea. Latest consensus state: current_term: 1 leader_uuid: "f0073c7b2bb64923b98689c6a46ccbea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0073c7b2bb64923b98689c6a46ccbea" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 41053 } } }
I20260809 13:02:14.004997 15252 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:14.005676 15251 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0073c7b2bb64923b98689c6a46ccbea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0073c7b2bb64923b98689c6a46ccbea" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 41053 } } }
I20260809 13:02:14.006063 15251 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:14.006552 15261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:02:14.011301 15261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:02:14.019959 15261 catalog_manager.cc:1347] Generated new cluster ID: 41a383622b7b49d5b0e5e6fde02ea6e9
I20260809 13:02:14.020089 15261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:02:14.050396 15261 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:02:14.050948 15261 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:02:14.059159 15261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0073c7b2bb64923b98689c6a46ccbea: Generated new TSK 0
I20260809 13:02:14.060374 15261 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:02:14.080073 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 15163
W20260809 13:02:14.081458 15232 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57713
W20260809 13:02:14.082244 22445 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 15163: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:02:14.082463 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 15163
2026-08-09T13:02:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2131 ms)
[----------] 18 tests from AuthTokenIssuingTest (38504 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:02:14 dist-test-slave-gp6n krb5kdc[15273](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:02:14 dist-test-slave-gp6n krb5kdc[15273](info): set up 2 sockets
Aug 09 13:02:14 dist-test-slave-gp6n krb5kdc[15273](info): commencing operation
krb5kdc: starting...
W20260809 13:02:15.289770 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:02:15 dist-test-slave-gp6n krb5kdc[15273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:02:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:02:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 13:02:15.622368 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:43325
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:33039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:43325,127.21.235.125:37811,127.21.235.124:36717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:02:15.973857 15289 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 13:02:15.974263 15289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:02:15.974435 15289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:02:15.974575 15289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:02:15.981999 15289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:02:15.982131 15289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:02:15.982246 15289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:02:15.982342 15289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:02:15.982457 15289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:02:15.992632 15289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:43325,127.21.235.125:37811,127.21.235.124:36717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:43325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:02:15.994017 15289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:02:15.995458 15289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:02:16.007684 15294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:16.007638 15295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:16.007711 15297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:02:16.008864 15289 server_base.cc:1034] running on GCE node
I20260809 13:02:16.009675 15289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:02:16.011236 15289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:02:16.012493 15289 hybrid_clock.cc:648] HybridClock initialized: now 1786280536012418 us; error 50 us; skew 500 ppm
I20260809 13:02:16.012848 15289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:02:16 dist-test-slave-gp6n krb5kdc[15273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280536, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:02:16.022540 15289 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 13:02:16.024336 15289 webserver.cc:460] Webserver started at http://127.21.235.126:46521/ using document root <none> and password file <none>
I20260809 13:02:16.025084 15289 fs_manager.cc:359] Metadata directory not provided
I20260809 13:02:16.025223 15289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:02:16.025589 15289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:02:16.028702 15289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "0fc54adb0d354251b9b31a85d9f19ab5"
format_stamp: "Formatted at 2026-08-09 13:02:16 on dist-test-slave-gp6n"
server_key: "9b3129775298d90e1713bdef41eda2bb"
server_key_iv: "a118f917f7a5b6a80c3b580bbc49a699"
server_key_version: "encryptionkey@0"
I20260809 13:02:16.029587 15289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "0fc54adb0d354251b9b31a85d9f19ab5"
format_stamp: "Formatted at 2026-08-09 13:02:16 on dist-test-slave-gp6n"
server_key: "9b3129775298d90e1713bdef41eda2bb"
server_key_iv: "a118f917f7a5b6a80c3b580bbc49a699"
server_key_version: "encryptionkey@0"
I20260809 13:02:16.035176 15289 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 13:02:16.038841 15305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:02:16.040364 15289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 13:02:16.040558 15289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "0fc54adb0d354251b9b31a85d9f19ab5"
format_stamp: "Formatted at 2026-08-09 13:02:16 on dist-test-slave-gp6n"
server_key: "9b3129775298d90e1713bdef41eda2bb"
server_key_iv: "a118f917f7a5b6a80c3b580bbc49a699"
server_key_version: "encryptionkey@0"
I20260809 13:02:16.040791 15289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:02:16.076676 15289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:02:16.081120 15289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:02:16.081413 15289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:02:16.100970 15289 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:43325
I20260809 13:02:16.100994 15362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:43325 every 8 connection(s)
I20260809 13:02:16.102414 15289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:02:16.106050 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 15289
I20260809 13:02:16.106299 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:02:16.106805 15363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:02:16.107195 22445 external_mini_cluster.cc:1428] Setting key b11b035d78b2f3243d3997c56bc78891
I20260809 13:02:16.117483 15363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:02:16 dist-test-slave-gp6n krb5kdc[15273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280536, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.235.125@KRBTEST.COM; defaulting to no policy
I20260809 13:02:16.144927 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.121886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:43325 (local address 127.0.0.1:55352)
0809 13:02:16.122961 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.123017 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 13:02:16.125096 (+ 2079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:02:16.126079 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:02:16.126084 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:02:16.126748 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:02:16.127107 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:16.127126 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:16.129561 (+ 2435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:16.129566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:16.130775 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:16.130788 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:16.132212 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:16.132217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:16.132345 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:16.133079 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:02:16.133116 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:02:16.135721 (+ 2605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:02:16.140526 (+ 4805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:16.140537 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:16.140548 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:16.140983 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:16.141509 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:16.141512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:16.141514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:16.141676 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:16.142771 (+ 1095us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:02:16.142778 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:02:16.143007 (+ 229us) client_negotiation.cc:769] Sending connection context
0809 13:02:16.143239 (+ 232us) client_negotiation.cc:240] Negotiation successful
0809 13:02:16.143676 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"thread_start_us":455,"threads_started":1}
I20260809 13:02:16.144964 15367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.121886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55352 (local address 127.21.235.126:43325)
0809 13:02:16.124140 (+ 2254us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:16.124151 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:16.124203 (+ 52us) server_negotiation.cc:407] Connection header received
0809 13:02:16.125509 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:16.125520 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:16.125832 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:16.126061 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:16.127355 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:16.129384 (+ 2029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:16.130934 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:16.131961 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:16.132203 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:16.136002 (+ 3799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:16.136017 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:16.136040 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:16.136094 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:16.140330 (+ 4236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:16.141148 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:16.141161 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:16.141167 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:16.141385 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:16.141802 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:16.141807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:16.141811 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:16.142620 (+ 809us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:16.142848 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:16.143478 (+ 630us) server_negotiation.cc:299] Negotiation successful
0809 13:02:16.143878 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":2016,"spinlock_wait_cycles":71424,"thread_start_us":1724,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.125@KRBTEST.COM" created.
I20260809 13:02:16.147926 15363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:02:16.149072 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.148552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:37811 (local address 127.0.0.1:60820)
0809 13:02:16.148764 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.148914 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:37811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 13:02:16.149362 15309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.235.125:37811: connect: Connection refused (error 111)
I20260809 13:02:16.151108 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.150624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:37811 (local address 127.0.0.1:60826)
0809 13:02:16.150803 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.150903 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:37811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 13:02:16.151355 15363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.125:37811: Network error: Client connection negotiation failed: client connection to 127.21.235.125:37811: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.125.keytab.
Entry for principal kudu/127.21.235.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.125.keytab.
I20260809 13:02:16.185977 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.125:37811
--webserver_interface=127.21.235.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.125.keytab
--principal=kudu/127.21.235.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:33039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:43325,127.21.235.125:37811,127.21.235.124:36717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:02:16.200701 15363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } attempt: 1
I20260809 13:02:16.201992 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.201487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:37811 (local address 127.0.0.1:60838)
0809 13:02:16.201691 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.201816 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:37811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 13:02:16.203157 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.202796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:37811 (local address 127.0.0.1:60846)
0809 13:02:16.202958 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.203027 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:37811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 13:02:16.203354 15363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.125:37811: Network error: Client connection negotiation failed: client connection to 127.21.235.125:37811: connect: Connection refused (error 111)
I20260809 13:02:16.271651 15363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } attempt: 2
I20260809 13:02:16.272969 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.272459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:37811 (local address 127.0.0.1:60848)
0809 13:02:16.272726 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.272821 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:37811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 13:02:16.274158 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.273753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:37811 (local address 127.0.0.1:60852)
0809 13:02:16.273927 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.274020 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:37811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260809 13:02:16.274453 15363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.125:37811: Network error: Client connection negotiation failed: client connection to 127.21.235.125:37811: connect: Connection refused (error 111)
I20260809 13:02:16.365759 15363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } attempt: 3
I20260809 13:02:16.367067 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.366606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:37811 (local address 127.0.0.1:60862)
0809 13:02:16.366809 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.366917 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:37811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 13:02:16.368194 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.367847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:37811 (local address 127.0.0.1:60878)
0809 13:02:16.367993 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.368064 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:37811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 13:02:16.368484 15363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.125:37811: Network error: Client connection negotiation failed: client connection to 127.21.235.125:37811: connect: Connection refused (error 111)
I20260809 13:02:16.511770 15363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } attempt: 4
I20260809 13:02:16.513170 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.512545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:37811 (local address 127.0.0.1:60884)
0809 13:02:16.512910 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.513017 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:37811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
I20260809 13:02:16.514489 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.513967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:37811 (local address 127.0.0.1:60892)
0809 13:02:16.514168 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.514284 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:37811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260809 13:02:16.514760 15363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.125:37811: Network error: Client connection negotiation failed: client connection to 127.21.235.125:37811: connect: Connection refused (error 111)
W20260809 13:02:16.539487 15370 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 13:02:16.539814 15370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:02:16.539940 15370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:02:16.540076 15370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:02:16.547482 15370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:02:16.547562 15370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:02:16.547660 15370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:02:16.547747 15370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:02:16.547859 15370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:02:16.557583 15370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:43325,127.21.235.125:37811,127.21.235.124:36717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.125.keytab
--principal=kudu/127.21.235.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.125:37811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.235.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:02:16.558902 15370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:02:16.560209 15370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:02:16.572077 15376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:16.572110 15375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:16.572018 15378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:02:16.573122 15370 server_base.cc:1034] running on GCE node
I20260809 13:02:16.573938 15370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:02:16.575433 15370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:02:16.576715 15370 hybrid_clock.cc:648] HybridClock initialized: now 1786280536576649 us; error 40 us; skew 500 ppm
I20260809 13:02:16.577085 15370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:02:16 dist-test-slave-gp6n krb5kdc[15273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280536, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:02:16.580396 15370 init.cc:377] Logged in from keytab as kudu/127.21.235.125@KRBTEST.COM (short username kudu)
I20260809 13:02:16.581787 15370 webserver.cc:460] Webserver started at http://127.21.235.125:44825/ using document root <none> and password file <none>
I20260809 13:02:16.582497 15370 fs_manager.cc:359] Metadata directory not provided
I20260809 13:02:16.582592 15370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:02:16.582855 15370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:02:16.585726 15370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-1/data/instance:
uuid: "34c4ce35f5d54307a1e154195d22315e"
format_stamp: "Formatted at 2026-08-09 13:02:16 on dist-test-slave-gp6n"
server_key: "23896708b027c49013eac665a1c6a27e"
server_key_iv: "63a438528a5192396a5a3d90475ef914"
server_key_version: "encryptionkey@0"
I20260809 13:02:16.586555 15370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-1/wal/instance:
uuid: "34c4ce35f5d54307a1e154195d22315e"
format_stamp: "Formatted at 2026-08-09 13:02:16 on dist-test-slave-gp6n"
server_key: "23896708b027c49013eac665a1c6a27e"
server_key_iv: "63a438528a5192396a5a3d90475ef914"
server_key_version: "encryptionkey@0"
I20260809 13:02:16.591711 15370 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 13:02:16.595047 15386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:02:16.596362 15370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:02:16.596545 15370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-1/wal
uuid: "34c4ce35f5d54307a1e154195d22315e"
format_stamp: "Formatted at 2026-08-09 13:02:16 on dist-test-slave-gp6n"
server_key: "23896708b027c49013eac665a1c6a27e"
server_key_iv: "63a438528a5192396a5a3d90475ef914"
server_key_version: "encryptionkey@0"
I20260809 13:02:16.596753 15370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:02:16.621062 15370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:02:16.625324 15370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:02:16.625609 15370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:02:16.644057 15370 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.125:37811
I20260809 13:02:16.644125 15446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.125:37811 every 8 connection(s)
I20260809 13:02:16.645493 15370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-1/data/info.pb
I20260809 13:02:16.649160 15447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:02:16.649255 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 15370
I20260809 13:02:16.649507 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-1/wal/instance
I20260809 13:02:16.649937 22445 external_mini_cluster.cc:1428] Setting key 09a34d229a0deeba39c0ec4f8bec8854
I20260809 13:02:16.655660 15447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:02:16 dist-test-slave-gp6n krb5kdc[15273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280536, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.125@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:02:16.678956 15449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.658213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55366 (local address 127.21.235.126:43325)
0809 13:02:16.658826 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:16.658835 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:16.659796 (+ 961us) server_negotiation.cc:407] Connection header received
0809 13:02:16.661665 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:16.661674 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:16.661768 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:16.661904 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:16.663564 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:16.664446 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:16.667016 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:16.667456 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:16.667553 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:16.672340 (+ 4787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:16.672345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:16.672349 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:16.672393 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:16.675212 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:16.676311 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:16.676316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:16.676318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:16.676450 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:16.676938 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:16.676941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:16.676943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:16.677222 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:16.677340 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:16.678291 (+ 951us) server_negotiation.cc:299] Negotiation successful
0809 13:02:16.678555 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":296,"threads_started":1}
I20260809 13:02:16.680490 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.658833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:43325 (local address 127.0.0.1:55366)
0809 13:02:16.659614 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.659665 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 13:02:16.661395 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:02:16.662113 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:02:16.662120 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:02:16.662714 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:02:16.663314 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:16.663331 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:16.664625 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:16.664633 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:16.666866 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:16.666884 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:16.667713 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:16.667725 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:16.668090 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:16.668870 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:02:16.668905 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:02:16.672077 (+ 3172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:02:16.675376 (+ 3299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:16.675388 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:16.675416 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:16.676168 (+ 752us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:16.676563 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:16.676571 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:16.676577 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:16.676819 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:16.677361 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:02:16.677372 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:02:16.677859 (+ 487us) client_negotiation.cc:769] Sending connection context
0809 13:02:16.678197 (+ 338us) client_negotiation.cc:240] Negotiation successful
0809 13:02:16.678830 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":333,"threads_started":1}
I20260809 13:02:16.682462 15447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.235.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.124@KRBTEST.COM" created.
Aug 09 13:02:16 dist-test-slave-gp6n krb5kdc[15273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280536, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.125@KRBTEST.COM for kudu/127.21.235.125@KRBTEST.COM
I20260809 13:02:16.698099 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.683439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60904 (local address 127.21.235.125:37811)
0809 13:02:16.684181 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:16.684187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:16.684211 (+ 24us) server_negotiation.cc:407] Connection header received
0809 13:02:16.684251 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:16.684257 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:16.684435 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:16.684548 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:16.685292 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:16.686548 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:16.687722 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:16.688689 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:16.688832 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:16.692309 (+ 3477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:16.692320 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:16.692332 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:16.692361 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:16.695148 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:16.695984 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:16.695991 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:16.695993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:16.696104 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:16.696539 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:16.696542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:16.696544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:16.696884 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:16.697035 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:16.697466 (+ 431us) server_negotiation.cc:299] Negotiation successful
0809 13:02:16.697805 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":177,"server-negotiator.queue_time_us":592,"thread_start_us":442,"threads_started":1}
I20260809 13:02:16.698182 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.683174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:37811 (local address 127.0.0.1:60904)
0809 13:02:16.683372 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.683390 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:02:16.683528 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:02:16.684650 (+ 1122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:02:16.684653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:02:16.684989 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:02:16.685139 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:16.685148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:16.686686 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:16.686694 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:16.687575 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:16.687587 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:16.688836 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:16.688840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:16.688998 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:16.689631 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:02:16.689648 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:02:16.692104 (+ 2456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:02:16.695281 (+ 3177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:16.695289 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:16.695296 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:16.695859 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:16.696214 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:16.696222 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:16.696228 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:16.696427 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:16.697013 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:02:16.697021 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:02:16.697187 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 13:02:16.697343 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 13:02:16.697690 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":2}
I20260809 13:02:16.700789 15447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:02:16.702078 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.701526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42728)
0809 13:02:16.701748 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.701882 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 13:02:16.702332 15390 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
I20260809 13:02:16.703991 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.703646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42738)
0809 13:02:16.703815 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.703890 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 13:02:16.704187 15447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:36717: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.124.keytab.
Entry for principal kudu/127.21.235.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.124.keytab.
I20260809 13:02:16.726249 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.124:36717
--webserver_interface=127.21.235.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.124.keytab
--principal=kudu/127.21.235.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:33039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:43325,127.21.235.125:37811,127.21.235.124:36717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:02:16.753491 15447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } attempt: 1
I20260809 13:02:16.754876 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.754189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42746)
0809 13:02:16.754494 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.754652 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260809 13:02:16.756099 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.755598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42756)
0809 13:02:16.755800 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.755916 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 13:02:16.756320 15447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:36717: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
I20260809 13:02:16.814050 15363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } attempt: 5
Aug 09 13:02:16 dist-test-slave-gp6n krb5kdc[15273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280536, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for kudu/127.21.235.125@KRBTEST.COM
I20260809 13:02:16.824519 15447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } attempt: 2
I20260809 13:02:16.825728 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.825172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42760)
0809 13:02:16.825390 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.825522 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 13:02:16.826769 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.826356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42770)
0809 13:02:16.826515 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.826594 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260809 13:02:16.827099 15447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:36717: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
I20260809 13:02:16.828948 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.814973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60920 (local address 127.21.235.125:37811)
0809 13:02:16.815214 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:16.815221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:16.815244 (+ 23us) server_negotiation.cc:407] Connection header received
0809 13:02:16.815352 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:16.815358 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:16.815430 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:16.815557 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:16.816393 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:16.817111 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:16.818321 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:16.818871 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:16.819009 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:16.822779 (+ 3770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:16.822786 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:16.822789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:16.822822 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:16.826147 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:16.827034 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:16.827041 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:16.827043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:16.827198 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:16.827625 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:16.827630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:16.827641 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:16.827980 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:16.828091 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:16.828594 (+ 503us) server_negotiation.cc:299] Negotiation successful
0809 13:02:16.828756 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 13:02:16.829203 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.814852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:37811 (local address 127.0.0.1:60920)
0809 13:02:16.815061 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.815087 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 13:02:16.815221 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:02:16.815604 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:02:16.815611 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:02:16.816028 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:02:16.816235 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:16.816244 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:16.817280 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:16.817284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:16.818165 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:16.818177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:16.819034 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:16.819039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:16.819257 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:16.819959 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:02:16.819991 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:02:16.822532 (+ 2541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:02:16.826327 (+ 3795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:16.826332 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:16.826336 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:16.826906 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:16.827305 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:16.827311 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:16.827314 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:16.827489 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:16.828105 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:02:16.828110 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:02:16.828308 (+ 198us) client_negotiation.cc:769] Sending connection context
0809 13:02:16.828501 (+ 193us) client_negotiation.cc:240] Negotiation successful
0809 13:02:16.828847 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 13:02:16.830304 15363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:02:16.831360 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.830896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42776)
0809 13:02:16.831096 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.831192 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 13:02:16.832432 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.832009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42786)
0809 13:02:16.832158 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.832268 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 13:02:16.832669 15363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:36717: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
I20260809 13:02:16.883957 15363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } attempt: 1
I20260809 13:02:16.885320 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.884736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42796)
0809 13:02:16.884974 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.885104 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260809 13:02:16.886478 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.886054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42808)
0809 13:02:16.886215 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.886305 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 13:02:16.886744 15363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:36717: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
I20260809 13:02:16.918354 15447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } attempt: 3
I20260809 13:02:16.919683 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.919216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42812)
0809 13:02:16.919419 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.919514 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 13:02:16.920938 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.920449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42826)
0809 13:02:16.920726 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.920803 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
W20260809 13:02:16.921159 15447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:36717: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
I20260809 13:02:16.955000 15363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } attempt: 2
I20260809 13:02:16.956334 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.955729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42838)
0809 13:02:16.955985 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.956117 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 13:02:16.957386 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:16.957012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42848)
0809 13:02:16.957163 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:16.957250 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 13:02:16.957609 15363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:36717: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
I20260809 13:02:17.063858 15363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } attempt: 3
I20260809 13:02:17.064354 15447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } attempt: 4
I20260809 13:02:17.065229 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.064653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42856)
0809 13:02:17.064891 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:17.065015 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 13:02:17.065498 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.064959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42870)
0809 13:02:17.065180 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:17.065271 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 13:02:17.066434 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.065936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42886)
0809 13:02:17.066189 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:17.066278 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260809 13:02:17.066591 15363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:36717: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
I20260809 13:02:17.066610 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.066201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42888)
0809 13:02:17.066344 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:17.066475 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 13:02:17.066864 15447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:36717: Network error: Client connection negotiation failed: client connection to 127.21.235.124:36717: connect: Connection refused (error 111)
W20260809 13:02:17.080542 15456 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 13:02:17.080843 15456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:02:17.080914 15456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:02:17.081000 15456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:02:17.088346 15456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:02:17.088434 15456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:02:17.088493 15456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:02:17.088552 15456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:02:17.088610 15456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:02:17.099283 15456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:33039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:43325,127.21.235.125:37811,127.21.235.124:36717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.124.keytab
--principal=kudu/127.21.235.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.124:36717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.235.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:02:17.100605 15456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:02:17.101994 15456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:02:17.114445 15461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:17.114341 15464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:17.114385 15462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:02:17.115293 15456 server_base.cc:1034] running on GCE node
I20260809 13:02:17.116144 15456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:02:17.117653 15456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:02:17.118899 15456 hybrid_clock.cc:648] HybridClock initialized: now 1786280537118841 us; error 29 us; skew 500 ppm
I20260809 13:02:17.119246 15456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:02:17 dist-test-slave-gp6n krb5kdc[15273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280537, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:02:17.122661 15456 init.cc:377] Logged in from keytab as kudu/127.21.235.124@KRBTEST.COM (short username kudu)
I20260809 13:02:17.124010 15456 webserver.cc:460] Webserver started at http://127.21.235.124:37415/ using document root <none> and password file <none>
I20260809 13:02:17.124663 15456 fs_manager.cc:359] Metadata directory not provided
I20260809 13:02:17.124748 15456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:02:17.125010 15456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:02:17.128139 15456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-2/data/instance:
uuid: "7ff69cbaacaa4262a5fe0109c4094d7f"
format_stamp: "Formatted at 2026-08-09 13:02:17 on dist-test-slave-gp6n"
server_key: "68613c9d98edbc5466dc5b3d27283d59"
server_key_iv: "cefe29b969b3cc6b048695c239ed9044"
server_key_version: "encryptionkey@0"
I20260809 13:02:17.128880 15456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-2/wal/instance:
uuid: "7ff69cbaacaa4262a5fe0109c4094d7f"
format_stamp: "Formatted at 2026-08-09 13:02:17 on dist-test-slave-gp6n"
server_key: "68613c9d98edbc5466dc5b3d27283d59"
server_key_iv: "cefe29b969b3cc6b048695c239ed9044"
server_key_version: "encryptionkey@0"
I20260809 13:02:17.133834 15456 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 13:02:17.137171 15471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:02:17.138523 15456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 13:02:17.138666 15456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-2/wal
uuid: "7ff69cbaacaa4262a5fe0109c4094d7f"
format_stamp: "Formatted at 2026-08-09 13:02:17 on dist-test-slave-gp6n"
server_key: "68613c9d98edbc5466dc5b3d27283d59"
server_key_iv: "cefe29b969b3cc6b048695c239ed9044"
server_key_version: "encryptionkey@0"
I20260809 13:02:17.138841 15456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:02:17.154481 15456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:02:17.159065 15456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:02:17.159297 15456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:02:17.178623 15456 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.124:36717
I20260809 13:02:17.178678 15529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.124:36717 every 8 connection(s)
I20260809 13:02:17.180033 15456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-2/data/info.pb
I20260809 13:02:17.183781 15532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:02:17.188364 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 15456
I20260809 13:02:17.188565 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280163688078-22445-0/minicluster-data/master-2/wal/instance
I20260809 13:02:17.189121 22445 external_mini_cluster.cc:1428] Setting key 424b16b7b2c7967e4cf671170d021773
I20260809 13:02:17.191278 15532 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:02:17 dist-test-slave-gp6n krb5kdc[15273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280537, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.124@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
Aug 09 13:02:17 dist-test-slave-gp6n krb5kdc[15273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.124@KRBTEST.COM
I20260809 13:02:17.212456 15536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.193796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55374 (local address 127.21.235.126:43325)
0809 13:02:17.194327 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.194336 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.194806 (+ 470us) server_negotiation.cc:407] Connection header received
0809 13:02:17.195065 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.195071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.195165 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.195286 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.197111 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.198225 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.200477 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.201126 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.201286 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.205492 (+ 4206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:17.205498 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:17.205501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:17.205544 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:17.208792 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.209770 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.209774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.209776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.209901 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.210362 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.210366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.210369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.210788 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:17.210938 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.211805 (+ 867us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.212026 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":263,"threads_started":1}
I20260809 13:02:17.213568 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.193883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:43325 (local address 127.0.0.1:55374)
0809 13:02:17.194670 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:17.194702 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 13:02:17.194882 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:02:17.195364 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:02:17.195371 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:02:17.196250 (+ 879us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:02:17.196909 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:17.196929 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.198386 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.198389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:17.200319 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:17.200337 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.201371 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.201382 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:17.201558 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.202294 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:02:17.202326 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:02:17.205292 (+ 2966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:02:17.208926 (+ 3634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:17.208935 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:17.208948 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:17.209643 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:17.210024 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:17.210031 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:17.210036 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:17.210224 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:17.210917 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:02:17.210927 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:02:17.211328 (+ 401us) client_negotiation.cc:769] Sending connection context
0809 13:02:17.211651 (+ 323us) client_negotiation.cc:240] Negotiation successful
0809 13:02:17.212167 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":3,"thread_start_us":407,"threads_started":1}
I20260809 13:02:17.213593 15535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.191093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42890 (local address 127.21.235.124:36717)
0809 13:02:17.191938 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.191949 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.191989 (+ 40us) server_negotiation.cc:407] Connection header received
0809 13:02:17.194082 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.194117 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.194670 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.195038 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.196822 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.198685 (+ 1863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.200406 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.201370 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.201554 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.206189 (+ 4635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:17.206198 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:17.206210 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:17.206237 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:17.209937 (+ 3700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.210690 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.210696 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.210698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.210818 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.211235 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.211240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.211243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.211654 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:17.211934 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.212303 (+ 369us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.212537 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":630,"thread_start_us":415,"threads_started":1}
I20260809 13:02:17.215302 15532 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:02:17 dist-test-slave-gp6n krb5kdc[15273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280537, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.124@KRBTEST.COM for kudu/127.21.235.125@KRBTEST.COM
Aug 09 13:02:17 dist-test-slave-gp6n krb5kdc[15273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:02:17.230791 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.216210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60926 (local address 127.21.235.125:37811)
0809 13:02:17.216390 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.216394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.216410 (+ 16us) server_negotiation.cc:407] Connection header received
0809 13:02:17.216458 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.216462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.216519 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.216598 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.217456 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.218144 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.219931 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.220575 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.220701 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.224431 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:17.224447 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:17.224455 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:17.224502 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:17.227704 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.228623 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.228627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.228629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.228762 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.229224 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.229233 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.229238 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.229619 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:17.229749 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.230254 (+ 505us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.230529 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 13:02:17.230808 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.215959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:37811 (local address 127.0.0.1:60926)
0809 13:02:17.216209 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:17.216229 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:02:17.216334 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:02:17.216668 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:02:17.216673 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:02:17.217054 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:02:17.217279 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:17.217292 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.218274 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.218278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:17.219782 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:17.219797 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.220719 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.220724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:17.220888 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.221479 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:02:17.221501 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:02:17.224200 (+ 2699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:02:17.227897 (+ 3697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:17.227902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:17.227907 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:17.228492 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:17.228877 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:17.228883 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:17.228888 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:17.229072 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:17.229760 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:02:17.229766 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:02:17.229935 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 13:02:17.230099 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 13:02:17.230375 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260809 13:02:17.231721 15532 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:02:17.235016 15536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.217379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55380 (local address 127.21.235.126:43325)
0809 13:02:17.218347 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.218351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.218363 (+ 12us) server_negotiation.cc:407] Connection header received
0809 13:02:17.218463 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.218468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.218543 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.218618 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.220138 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.221117 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.223014 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.223701 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.223796 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.228426 (+ 4630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:17.228433 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:17.228438 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:17.228475 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:17.231873 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.233108 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.233112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.233114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.233229 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.233609 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.233612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.233614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.233965 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:17.234279 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.234470 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.234692 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":876}
Aug 09 13:02:17 dist-test-slave-gp6n krb5kdc[15273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280537, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.124@KRBTEST.COM for kudu/127.21.235.124@KRBTEST.COM
I20260809 13:02:17.243803 15363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } attempt: 4
Aug 09 13:02:17 dist-test-slave-gp6n krb5kdc[15273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.125@KRBTEST.COM
I20260809 13:02:17.246918 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.232410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42898)
0809 13:02:17.232593 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:17.232611 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:02:17.232705 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:02:17.233011 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:02:17.233018 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:02:17.233482 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:02:17.233684 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:17.233693 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.234961 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.234965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:17.235963 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:17.235975 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.237003 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.237009 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:17.237265 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.237851 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:02:17.237866 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:02:17.240090 (+ 2224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:02:17.243495 (+ 3405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:17.243502 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:17.243507 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:17.244086 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:17.244618 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:17.244623 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:17.244627 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:17.244890 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:17.245747 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:02:17.245754 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:02:17.245956 (+ 202us) client_negotiation.cc:769] Sending connection context
0809 13:02:17.246162 (+ 206us) client_negotiation.cc:240] Negotiation successful
0809 13:02:17.246502 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 13:02:17.247110 15535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.232637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42898 (local address 127.21.235.124:36717)
0809 13:02:17.232777 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.232782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.232795 (+ 13us) server_negotiation.cc:407] Connection header received
0809 13:02:17.232830 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.232833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.232882 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.232954 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.233819 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.234852 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.236105 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.236522 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.236655 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.240261 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:17.240266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:17.240269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:17.240297 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:17.243356 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.244301 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.244306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.244310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.244480 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.245033 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.245039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.245042 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.245527 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:17.245655 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.246273 (+ 618us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.246673 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":159,"server-negotiator.queue_time_us":54}
I20260809 13:02:17.253022 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.236713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60940 (local address 127.21.235.125:37811)
0809 13:02:17.236899 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.236908 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.236926 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:02:17.236983 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.236988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.237058 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.237157 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.238490 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.239382 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.240905 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.241303 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.241620 (+ 317us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.246889 (+ 5269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:17.246899 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:17.246905 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:17.246950 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:17.249979 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.250687 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.250693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.250696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.250854 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.251252 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.251257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.251261 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.251531 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:17.251861 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.252440 (+ 579us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.252679 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 13:02:17 dist-test-slave-gp6n krb5kdc[15273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280536, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for kudu/127.21.235.124@KRBTEST.COM
I20260809 13:02:17.254081 15532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f: Bootstrap starting.
I20260809 13:02:17.258149 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:02:17.258772 15532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f: Neither blocks nor log segments found. Creating new log.
I20260809 13:02:17.260080 15532 log.cc:824] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f: Log is configured to *not* fsync() on all Append() calls
I20260809 13:02:17.262044 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.244413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42910)
0809 13:02:17.244826 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:17.244847 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:02:17.244961 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:02:17.245901 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:02:17.245909 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:02:17.246455 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:02:17.246672 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:17.246686 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.248603 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.248608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:17.250024 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:17.250040 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.251099 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.251106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:17.251327 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.251952 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:02:17.251976 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:02:17.254663 (+ 2687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:02:17.259010 (+ 4347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:17.259016 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:17.259020 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:17.259544 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:17.259994 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:17.259999 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:17.260003 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:17.260171 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:17.260984 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:02:17.260989 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:02:17.261159 (+ 170us) client_negotiation.cc:769] Sending connection context
0809 13:02:17.261317 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 13:02:17.261586 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290}
I20260809 13:02:17.262543 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.244778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42910 (local address 127.21.235.124:36717)
0809 13:02:17.245528 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.245532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.245550 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:02:17.245585 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.245588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.245737 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.245864 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.246874 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.248299 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.250208 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.250892 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.251619 (+ 727us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.254939 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:17.254946 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:17.254951 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:17.254986 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:17.258818 (+ 3832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.259678 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.259684 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.259690 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.259869 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.260319 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.260324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.260327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.260761 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:17.261818 (+ 1057us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.261911 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.262069 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":461,"threads_started":1}
I20260809 13:02:17.261726 15333 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0fc54adb0d354251b9b31a85d9f19ab5"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55380
I20260809 13:02:17.267999 15532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f: No bootstrap required, opened a new log
I20260809 13:02:17.268428 15363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5: Bootstrap starting.
I20260809 13:02:17.272511 15363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5: Neither blocks nor log segments found. Creating new log.
I20260809 13:02:17.273797 15363 log.cc:824] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5: Log is configured to *not* fsync() on all Append() calls
I20260809 13:02:17.277264 15363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5: No bootstrap required, opened a new log
I20260809 13:02:17.278594 15532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } } peers { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } } peers { permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } }
I20260809 13:02:17.279054 15532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ff69cbaacaa4262a5fe0109c4094d7f, State: Initialized, Role: FOLLOWER
I20260809 13:02:17.279630 15532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } } peers { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } } peers { permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } }
I20260809 13:02:17.280616 15541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } } peers { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } } peers { permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } } }
I20260809 13:02:17.281090 15541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:02:17.281885 15532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:02:17.285393 15363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } } peers { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } } peers { permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } }
I20260809 13:02:17.285854 15363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fc54adb0d354251b9b31a85d9f19ab5, State: Initialized, Role: FOLLOWER
I20260809 13:02:17.286562 15363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } } peers { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } } peers { permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } }
I20260809 13:02:17.287611 15553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } } peers { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } } peers { permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } } }
I20260809 13:02:17.288007 15553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:02:17.289314 15363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:02:17.289382 15333 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 13:02:17.289609 15333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:02:17.292981 15333 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } } peers { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } } peers { permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } }
I20260809 13:02:17.293371 15552 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 13:02:17.295073 15333 leader_election.cc:288] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [CANDIDATE]: Term 1 election: Requested vote from peers 34c4ce35f5d54307a1e154195d22315e (127.21.235.125:37811), 7ff69cbaacaa4262a5fe0109c4094d7f (127.21.235.124:36717)
I20260809 13:02:17.296372 15501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f"
I20260809 13:02:17.296872 15501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:02:17.301455 15501 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fc54adb0d354251b9b31a85d9f19ab5 in term 1.
I20260809 13:02:17.302301 15309 leader_election.cc:302] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fc54adb0d354251b9b31a85d9f19ab5, 7ff69cbaacaa4262a5fe0109c4094d7f; no voters:
I20260809 13:02:17.303072 15553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:02:17.296995 15456 master_runner.cc:186] Error getting master registration for 127.21.235.126:43325: OK, instance_id { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" instance_seqno: 1786280536095590 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 13:02:17.312522 15456 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 13:02:17.313017 15553 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [term 1 LEADER]: Becoming Leader. State: Replica: 0fc54adb0d354251b9b31a85d9f19ab5, State: Running, Role: LEADER
I20260809 13:02:17.314265 15553 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } } peers { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } } peers { permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } }
I20260809 13:02:17.319139 15566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fc54adb0d354251b9b31a85d9f19ab5. Latest consensus state: current_term: 1 leader_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } } peers { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } } peers { permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } } }
I20260809 13:02:17.319602 15566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:17.321123 15574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260809 13:02:17.334337 15311 tablet.cc:2369] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:02:17.337102 15536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.317570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55392 (local address 127.21.235.126:43325)
0809 13:02:17.317818 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.317828 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.318974 (+ 1146us) server_negotiation.cc:407] Connection header received
0809 13:02:17.319268 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.319273 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.319370 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.319456 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.321781 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.322962 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.324959 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.325726 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.325838 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.329543 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:17.329550 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:17.329554 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:17.329606 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:17.333604 (+ 3998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.334504 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.334509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.334512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.334682 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.335180 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.335185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.335187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.335573 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:17.335863 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.336404 (+ 541us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.336662 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 13:02:17.325024 15289 master_runner.cc:186] Error getting master registration for 127.21.235.125:37811: OK, instance_id { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" instance_seqno: 1786280536638643 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 13:02:17.338604 15289 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 13:02:17.346915 15574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:02:17.354233 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.327869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42918 (local address 127.21.235.124:36717)
0809 13:02:17.328105 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.328114 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.333299 (+ 5185us) server_negotiation.cc:407] Connection header received
0809 13:02:17.333741 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.333748 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.333872 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.333959 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.340198 (+ 6239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.341229 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.343316 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.344018 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.344136 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.347527 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:17.347535 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:17.347540 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:17.347583 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:17.351052 (+ 3469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.351925 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.351931 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.351935 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.352080 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.352524 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.352529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.352532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.352892 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:17.353112 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.353672 (+ 560us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.353905 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 13:02:17.356099 15501 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f [term 1 FOLLOWER]: Refusing update from remote peer 0fc54adb0d354251b9b31a85d9f19ab5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:02:17.357018 15566 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:02:17.365360 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.319307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60944 (local address 127.21.235.125:37811)
0809 13:02:17.319538 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.319545 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.327164 (+ 7619us) server_negotiation.cc:407] Connection header received
0809 13:02:17.328870 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.328880 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.328981 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.329076 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.334516 (+ 5440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.335885 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.340783 (+ 4898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.341822 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.341955 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.356970 (+ 15015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:17.356977 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:17.356982 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:17.357029 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:17.360719 (+ 3690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.361576 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.361583 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.361587 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.361762 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.362253 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.362262 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.362268 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.362789 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:17.364298 (+ 1509us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.364486 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.364806 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260809 13:02:17.366082 15447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } attempt: 5
Aug 09 13:02:17 dist-test-slave-gp6n krb5kdc[15273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280536, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.125@KRBTEST.COM for kudu/127.21.235.124@KRBTEST.COM
I20260809 13:02:17.380029 15541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fc54adb0d354251b9b31a85d9f19ab5. Latest consensus state: current_term: 1 leader_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } } peers { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } } peers { permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } } }
I20260809 13:02:17.380380 15541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:02:17.382534 15578 mvcc.cc:205] Tried to move back new op lower bound from 7316605081000071168 to 7316605080846020608. Current Snapshot: MvccSnapshot[applied={T|T < 7316605081000071168}]
I20260809 13:02:17.390182 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.366637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:36717 (local address 127.0.0.1:42930)
0809 13:02:17.366798 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:17.366816 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:02:17.366921 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:02:17.367414 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:02:17.367422 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:02:17.368065 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:02:17.368338 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:17.368353 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.370459 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.370468 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:17.372223 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:17.372244 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.373496 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.373502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:17.373751 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.374450 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:02:17.374474 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:02:17.377248 (+ 2774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:02:17.386863 (+ 9615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:17.386880 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:17.386886 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:17.387415 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:17.388141 (+ 726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:17.388147 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:17.388150 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:17.388357 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:17.389080 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:02:17.389086 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:02:17.389259 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 13:02:17.389437 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 13:02:17.389754 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 13:02:17.391351 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.366658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42930 (local address 127.21.235.124:36717)
0809 13:02:17.366828 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.366834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.366858 (+ 24us) server_negotiation.cc:407] Connection header received
0809 13:02:17.367068 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.367077 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.367174 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.368535 (+ 1361us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.368718 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.370229 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.372479 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.373324 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.373486 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.377507 (+ 4021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:17.377513 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:17.377517 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:17.377557 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:17.386700 (+ 9143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.387585 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.387593 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.387598 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.387779 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:17.388517 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:17.388521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:17.388524 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:17.388932 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:17.390593 (+ 1661us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.390743 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.390991 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 13:02:17.391963 15566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } } peers { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } } peers { permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } } }
I20260809 13:02:17.392074 15541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } } peers { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } } peers { permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } } }
I20260809 13:02:17.392282 15566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:17.392318 15541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:02:17.393146 15574 catalog_manager.cc:1347] Generated new cluster ID: 2c9c3a90101e4ecbaa1820ef584a4e50
I20260809 13:02:17.393241 15574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:02:17.398689 15447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e: Bootstrap starting.
I20260809 13:02:17.403503 15447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e: Neither blocks nor log segments found. Creating new log.
I20260809 13:02:17.404835 15447 log.cc:824] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e: Log is configured to *not* fsync() on all Append() calls
I20260809 13:02:17.409381 15574 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:02:17.410254 15447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e: No bootstrap required, opened a new log
I20260809 13:02:17.411300 15574 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:02:17.422626 15447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } } peers { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } } peers { permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } }
I20260809 13:02:17.423130 15447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c4ce35f5d54307a1e154195d22315e, State: Initialized, Role: FOLLOWER
I20260809 13:02:17.423727 15447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } } peers { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } } peers { permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } }
I20260809 13:02:17.424310 15574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5: Generated new TSK 0
I20260809 13:02:17.426679 15574 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:02:17.431012 15588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } } peers { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } } peers { permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } } }
I20260809 13:02:17.431449 15588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:02:17.432235 15447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:02:17.432333 15418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "34c4ce35f5d54307a1e154195d22315e"
I20260809 13:02:17.432844 15418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:02:17.436954 15418 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fc54adb0d354251b9b31a85d9f19ab5 in term 1.
I20260809 13:02:17.437909 15417 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e [term 1 FOLLOWER]: Refusing update from remote peer 0fc54adb0d354251b9b31a85d9f19ab5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:02:17.444386 15553 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:02:17.481495 15588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fc54adb0d354251b9b31a85d9f19ab5. Latest consensus state: current_term: 1 leader_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } } peers { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } } peers { permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } } }
I20260809 13:02:17.481879 15588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:02:17.482352 15599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } } peers { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } } peers { permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } } }
I20260809 13:02:17.482892 15599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:02:17.501786 15606 catalog_manager.cc:1259] Loaded cluster ID: 2c9c3a90101e4ecbaa1820ef584a4e50
I20260809 13:02:17.501988 15606 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e: loading cluster ID for follower catalog manager: success
I20260809 13:02:17.506546 15606 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e: acquiring CA information for follower catalog manager: success
I20260809 13:02:17.513396 15606 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 34c4ce35f5d54307a1e154195d22315e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 13:02:17.536902 15536 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:17.533695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55398 (local address 127.21.235.126:43325)
0809 13:02:17.533949 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.533958 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.534609 (+ 651us) server_negotiation.cc:407] Connection header received
0809 13:02:17.534928 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.534934 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.535028 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.535111 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.536550 (+ 1439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55398: BlockingRecv error: recv got EOF from 127.0.0.1:55398 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260809 13:02:17.540014 15453 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:17.534744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60954 (local address 127.21.235.125:37811)
0809 13:02:17.534950 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.534956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.537947 (+ 2991us) server_negotiation.cc:407] Connection header received
0809 13:02:17.538243 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.538252 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.538356 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.538470 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.539668 (+ 1198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60954: BlockingRecv error: recv got EOF from 127.0.0.1:60954 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 13:02:17.543362 15538 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:17.541278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42942 (local address 127.21.235.124:36717)
0809 13:02:17.541465 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.541469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.541698 (+ 229us) server_negotiation.cc:407] Connection header received
0809 13:02:17.541882 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.541888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.541997 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.542117 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.543117 (+ 1000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42942: BlockingRecv error: recv got EOF from 127.0.0.1:42942 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 13:02:17.554806 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.547487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60960 (local address 127.21.235.125:37811)
0809 13:02:17.547637 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.547643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.549392 (+ 1749us) server_negotiation.cc:407] Connection header received
0809 13:02:17.549610 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.549616 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.549781 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.549949 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:17.550356 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.551173 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.552499 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.552944 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.553097 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.553579 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:17.554099 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:17.554290 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.554340 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.554460 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 13:02:17.564363 15538 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:17.561505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42946 (local address 127.21.235.124:36717)
0809 13:02:17.561756 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.561765 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.562943 (+ 1178us) server_negotiation.cc:407] Connection header received
0809 13:02:17.563180 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.563184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.563278 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.563400 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.564137 (+ 737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42946: BlockingRecv error: recv got EOF from 127.0.0.1:42946 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 13:02:17.574270 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.568912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60964 (local address 127.21.235.125:37811)
0809 13:02:17.569154 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.569162 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.569664 (+ 502us) server_negotiation.cc:407] Connection header received
0809 13:02:17.569852 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.569858 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.569948 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.570046 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:17.570466 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.571227 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.572540 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.573008 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.573153 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.573540 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:17.573674 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:17.573777 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.573928 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.574008 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 13:02:17.588464 15538 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:17.585857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42958 (local address 127.21.235.124:36717)
0809 13:02:17.586090 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.586100 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.586977 (+ 877us) server_negotiation.cc:407] Connection header received
0809 13:02:17.587186 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.587192 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.587272 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.587388 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.588173 (+ 785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42958: BlockingRecv error: recv got EOF from 127.0.0.1:42958 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 13:02:17.599084 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.593476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60970 (local address 127.21.235.125:37811)
0809 13:02:17.593733 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.593739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.594282 (+ 543us) server_negotiation.cc:407] Connection header received
0809 13:02:17.594522 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.594528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.594607 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.594708 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:17.595152 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.595906 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.597199 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.597696 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.597847 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.598241 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:17.598377 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:17.598511 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.598662 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.598743 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
W20260809 13:02:17.608283 15538 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:17.604862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42962 (local address 127.21.235.124:36717)
0809 13:02:17.605100 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.605110 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.606964 (+ 1854us) server_negotiation.cc:407] Connection header received
0809 13:02:17.607157 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.607163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.607247 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.607352 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.608050 (+ 698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42962: BlockingRecv error: recv got EOF from 127.0.0.1:42962 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 13:02:17.620267 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.614826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60982 (local address 127.21.235.125:37811)
0809 13:02:17.615069 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.615077 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.615557 (+ 480us) server_negotiation.cc:407] Connection header received
0809 13:02:17.615751 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.615756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.615832 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.615928 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:17.616301 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.617072 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.618514 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.618969 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.619109 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.619517 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:17.619622 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:17.619721 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.619866 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.619951 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 13:02:17.628468 15538 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:17.626143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42974 (local address 127.21.235.124:36717)
0809 13:02:17.626527 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.626537 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.627064 (+ 527us) server_negotiation.cc:407] Connection header received
0809 13:02:17.627284 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.627290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.627372 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.627483 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.628253 (+ 770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42974: BlockingRecv error: recv got EOF from 127.0.0.1:42974 (error 108)
Metrics: {"server-negotiator.queue_time_us":180}
I20260809 13:02:17.647495 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.639699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60990 (local address 127.21.235.125:37811)
0809 13:02:17.639936 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.639946 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.640496 (+ 550us) server_negotiation.cc:407] Connection header received
0809 13:02:17.640726 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.640731 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.640810 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.640926 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:17.641354 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.642212 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.644998 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.645731 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.645877 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.646628 (+ 751us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:17.646777 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:17.646853 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.647078 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.647161 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 13:02:17.689338 15538 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:17.686924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42980 (local address 127.21.235.124:36717)
0809 13:02:17.687130 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.687136 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.687813 (+ 677us) server_negotiation.cc:407] Connection header received
0809 13:02:17.688049 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.688054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.688144 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.688259 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.689061 (+ 802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42980: BlockingRecv error: recv got EOF from 127.0.0.1:42980 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 13:02:17.717660 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.711776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60992 (local address 127.21.235.125:37811)
0809 13:02:17.712052 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.712061 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.712611 (+ 550us) server_negotiation.cc:407] Connection header received
0809 13:02:17.712848 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.712853 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.712928 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.713054 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:17.713554 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.714369 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.715789 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.716230 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.716383 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.716838 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:17.716993 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:17.717100 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.717255 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.717354 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
W20260809 13:02:17.728428 15538 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:17.723874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42990 (local address 127.21.235.124:36717)
0809 13:02:17.724126 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.724135 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.726982 (+ 2847us) server_negotiation.cc:407] Connection header received
0809 13:02:17.727189 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.727194 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.727267 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.727375 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.728165 (+ 790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42990: BlockingRecv error: recv got EOF from 127.0.0.1:42990 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 13:02:17.769537 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.763627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32770 (local address 127.21.235.125:37811)
0809 13:02:17.763896 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.763906 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.764461 (+ 555us) server_negotiation.cc:407] Connection header received
0809 13:02:17.764684 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.764689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.764758 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.764860 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:17.765342 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.766097 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.767525 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.768020 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.768168 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.768731 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:17.768832 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:17.768938 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.769093 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.769171 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 13:02:17.777935 15538 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:17.775351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42992 (local address 127.21.235.124:36717)
0809 13:02:17.775560 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.775567 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.776582 (+ 1015us) server_negotiation.cc:407] Connection header received
0809 13:02:17.776785 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.776792 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.776882 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.776999 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.777741 (+ 742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42992: BlockingRecv error: recv got EOF from 127.0.0.1:42992 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 13:02:17.853065 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.847323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32786 (local address 127.21.235.125:37811)
0809 13:02:17.847572 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.847583 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.848159 (+ 576us) server_negotiation.cc:407] Connection header received
0809 13:02:17.848418 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.848424 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.848496 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.848610 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:17.849030 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.849871 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.851279 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.851727 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.851868 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:17.852271 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:17.852380 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:17.852492 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:17.852636 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 13:02:17.852724 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 13:02:17.862744 15538 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:17.860270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43006 (local address 127.21.235.124:36717)
0809 13:02:17.860462 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.860471 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.861354 (+ 883us) server_negotiation.cc:407] Connection header received
0809 13:02:17.861577 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.861583 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.861665 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.861778 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:17.862554 (+ 776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43006: BlockingRecv error: recv got EOF from 127.0.0.1:43006 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 13:02:18.001719 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:17.995278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32788 (local address 127.21.235.125:37811)
0809 13:02:17.995537 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:17.995544 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:17.995928 (+ 384us) server_negotiation.cc:407] Connection header received
0809 13:02:17.996150 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:17.996155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:17.996224 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:17.996305 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:17.996836 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:17.997649 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:17.999723 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:18.000212 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:18.000345 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:18.000803 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:18.000922 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:18.001090 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:18.001294 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 13:02:18.001377 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 13:02:18.009627 15538 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:18.007724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43020 (local address 127.21.235.124:36717)
0809 13:02:18.007915 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:18.007921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:18.008237 (+ 316us) server_negotiation.cc:407] Connection header received
0809 13:02:18.008394 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:18.008400 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:18.008493 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:18.008612 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:18.009366 (+ 754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43020: BlockingRecv error: recv got EOF from 127.0.0.1:43020 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 13:02:18.048327 15553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc54adb0d354251b9b31a85d9f19ab5" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 43325 } } peers { permanent_uuid: "34c4ce35f5d54307a1e154195d22315e" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 37811 } } peers { permanent_uuid: "7ff69cbaacaa4262a5fe0109c4094d7f" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 36717 } } }
I20260809 13:02:18.048717 15553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fc54adb0d354251b9b31a85d9f19ab5 [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:18.274386 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:18.269340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32798 (local address 127.21.235.125:37811)
0809 13:02:18.269569 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:18.269576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:18.269738 (+ 162us) server_negotiation.cc:407] Connection header received
0809 13:02:18.269994 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:18.270000 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:18.270070 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:18.270172 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:18.270684 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:18.271414 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:18.272727 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:18.273149 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:18.273285 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:18.273713 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:18.273816 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:18.273918 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:18.274039 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 13:02:18.274120 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260809 13:02:18.281562 15538 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:18.279913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43026 (local address 127.21.235.124:36717)
0809 13:02:18.280088 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:18.280092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:18.280344 (+ 252us) server_negotiation.cc:407] Connection header received
0809 13:02:18.280489 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:18.280493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:18.280576 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:18.280658 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:18.281341 (+ 683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43026: BlockingRecv error: recv got EOF from 127.0.0.1:43026 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 13:02:18.297870 15552 catalog_manager.cc:1259] Loaded cluster ID: 2c9c3a90101e4ecbaa1820ef584a4e50
I20260809 13:02:18.298030 15552 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f: loading cluster ID for follower catalog manager: success
I20260809 13:02:18.300881 15552 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f: acquiring CA information for follower catalog manager: success
I20260809 13:02:18.302814 15552 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7ff69cbaacaa4262a5fe0109c4094d7f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 13:02:18.802548 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:18.797077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32806 (local address 127.21.235.125:37811)
0809 13:02:18.797790 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:18.797795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:18.797813 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:02:18.797879 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:18.797883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:18.797950 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:18.798042 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:18.798465 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:18.799210 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:18.800886 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:18.801363 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:18.801459 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:18.801873 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:18.801975 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:18.802055 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:18.802186 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 13:02:18.802246 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":488,"threads_started":1}
I20260809 13:02:18.814639 15740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:18.809217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43034 (local address 127.21.235.124:36717)
0809 13:02:18.809708 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:18.809712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:18.809730 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:02:18.809845 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:18.809849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:18.810001 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:18.810126 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:18.810570 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:18.811268 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:18.812487 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:18.812897 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:18.812989 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:18.813530 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:18.814060 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:18.814173 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:18.814307 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 13:02:18.814372 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":296,"threads_started":1}
I20260809 13:02:18.829373 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:18.821645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32812 (local address 127.21.235.125:37811)
0809 13:02:18.821866 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:18.821875 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:18.822298 (+ 423us) server_negotiation.cc:407] Connection header received
0809 13:02:18.822781 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:18.822791 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:18.822894 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:18.822987 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:18.823688 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:18.824707 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:18.827245 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:18.827933 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:18.828034 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:18.828512 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:18.828633 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:18.828734 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:18.828914 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 13:02:18.829005 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 13:02:18.829879 15748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:18.821027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55410 (local address 127.21.235.126:43325)
0809 13:02:18.821542 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:18.821549 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:18.821569 (+ 20us) server_negotiation.cc:407] Connection header received
0809 13:02:18.821818 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:18.821827 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:18.822075 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:18.822829 (+ 754us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:18.823040 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:18.824196 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:18.826930 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:18.827619 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:18.827759 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:18.828304 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:18.828776 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:18.829396 (+ 620us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:18.829458 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 13:02:18.829558 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":244,"threads_started":1}
I20260809 13:02:18.831462 15740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:18.822312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43036 (local address 127.21.235.124:36717)
0809 13:02:18.822511 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:18.822517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:18.823283 (+ 766us) server_negotiation.cc:407] Connection header received
0809 13:02:18.823476 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:18.823484 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:18.823568 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:18.823655 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:18.825155 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:18.825834 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:18.828219 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:18.828773 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:18.828893 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:18.830134 (+ 1241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:18.830253 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:18.830345 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:18.831069 (+ 724us) server_negotiation.cc:299] Negotiation successful
0809 13:02:18.831177 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 13:02:18.835799 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 15289
I20260809 13:02:18.836872 15357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:02:18.949782 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 15289
W20260809 13:02:18.976224 15475 connection.cc:535] server connection from 127.0.0.1:42910 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 13:02:18.976706 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 15370
I20260809 13:02:18.977887 15441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:02:19.086859 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 15370
I20260809 13:02:19.110059 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 15456
I20260809 13:02:19.111310 15524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:02:19.220077 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 15456
2026-08-09T13:02:19Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5143 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:02:19 dist-test-slave-gp6n krb5kdc[15756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:02:19 dist-test-slave-gp6n krb5kdc[15756](info): set up 2 sockets
Aug 09 13:02:19 dist-test-slave-gp6n krb5kdc[15756](info): commencing operation
krb5kdc: starting...
W20260809 13:02:20.453075 22445 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.002s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/test-user.keytab.
Aug 09 13:02:20 dist-test-slave-gp6n krb5kdc[15756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:02:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:02:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
Entry for principal kudu/127.21.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab.
I20260809 13:02:20.794265 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:46173
--webserver_interface=127.21.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:46173,127.21.235.125:44027,127.21.235.124:41639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:02:21.150960 15774 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 13:02:21.151403 15774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:02:21.151530 15774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:02:21.151666 15774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:02:21.159293 15774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:02:21.159395 15774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:02:21.159493 15774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:02:21.159616 15774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:02:21.159713 15774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:02:21.170069 15774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:46173,127.21.235.125:44027,127.21.235.124:41639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.126.keytab
--principal=kudu/127.21.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.126:46173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:02:21.171483 15774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:02:21.172888 15774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:02:21.185420 15780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:21.185482 15783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:21.185524 15781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:02:21.186280 15774 server_base.cc:1034] running on GCE node
I20260809 13:02:21.187091 15774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:02:21.188525 15774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:02:21.189774 15774 hybrid_clock.cc:648] HybridClock initialized: now 1786280541189682 us; error 64 us; skew 500 ppm
I20260809 13:02:21.190138 15774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:02:21 dist-test-slave-gp6n krb5kdc[15756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280541, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:02:21.196902 15774 init.cc:377] Logged in from keytab as kudu/127.21.235.126@KRBTEST.COM (short username kudu)
I20260809 13:02:21.198388 15774 webserver.cc:460] Webserver started at http://127.21.235.126:45237/ using document root <none> and password file <none>
I20260809 13:02:21.199200 15774 fs_manager.cc:359] Metadata directory not provided
I20260809 13:02:21.199335 15774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:02:21.199671 15774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:02:21.202736 15774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-0/data/instance:
uuid: "602c8155050244319e250864f6aaa80f"
format_stamp: "Formatted at 2026-08-09 13:02:21 on dist-test-slave-gp6n"
server_key: "51c5b0b5adb6ec6c088a1ec1b1f4495a"
server_key_iv: "41c08a4939a0e422f8427894c7893ad7"
server_key_version: "encryptionkey@0"
I20260809 13:02:21.203573 15774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-0/wal/instance:
uuid: "602c8155050244319e250864f6aaa80f"
format_stamp: "Formatted at 2026-08-09 13:02:21 on dist-test-slave-gp6n"
server_key: "51c5b0b5adb6ec6c088a1ec1b1f4495a"
server_key_iv: "41c08a4939a0e422f8427894c7893ad7"
server_key_version: "encryptionkey@0"
I20260809 13:02:21.209025 15774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:02:21.212608 15790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:02:21.214082 15774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:02:21.214303 15774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-0/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-0/wal
uuid: "602c8155050244319e250864f6aaa80f"
format_stamp: "Formatted at 2026-08-09 13:02:21 on dist-test-slave-gp6n"
server_key: "51c5b0b5adb6ec6c088a1ec1b1f4495a"
server_key_iv: "41c08a4939a0e422f8427894c7893ad7"
server_key_version: "encryptionkey@0"
I20260809 13:02:21.214648 15774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:02:21.252172 15774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:02:21.256461 15774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:02:21.256773 15774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:02:21.276695 15774 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.126:46173
I20260809 13:02:21.276734 15849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.126:46173 every 8 connection(s)
I20260809 13:02:21.278113 15774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-0/data/info.pb
I20260809 13:02:21.282121 15850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:02:21.288564 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 15774
I20260809 13:02:21.288787 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-0/wal/instance
I20260809 13:02:21.289368 22445 external_mini_cluster.cc:1428] Setting key 7bef9a9f879cc64622a034eb9bde6370
I20260809 13:02:21.289084 15850 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:02:21 dist-test-slave-gp6n krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280541, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:02:21.312250 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.292860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54248 (local address 127.21.235.126:46173)
0809 13:02:21.294607 (+ 1747us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:21.294615 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:21.294664 (+ 49us) server_negotiation.cc:407] Connection header received
0809 13:02:21.296178 (+ 1514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:21.296184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:21.296410 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:21.296616 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:21.297910 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:21.299188 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:21.300553 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:21.301103 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:21.301307 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:21.304843 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:21.304851 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:21.304862 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:21.304891 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:21.307760 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:21.308441 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:21.308450 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:21.308459 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:21.308598 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:21.309000 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:21.309008 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:21.309011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:21.309507 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:21.309723 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:21.310286 (+ 563us) server_negotiation.cc:299] Negotiation successful
0809 13:02:21.310743 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":1489,"thread_start_us":1263,"threads_started":1}
I20260809 13:02:21.312282 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.292860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:46173 (local address 127.0.0.1:54248)
0809 13:02:21.293719 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.293780 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 13:02:21.295816 (+ 2036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:02:21.296636 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:02:21.296643 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:02:21.297356 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:02:21.297698 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:21.297715 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:21.299345 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:21.299350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:21.300412 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:21.300424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:21.301339 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:21.301346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:21.301532 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:21.302246 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:02:21.302272 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:02:21.304604 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:02:21.307894 (+ 3290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:21.307907 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:21.307919 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:21.308308 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:21.308716 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:21.308722 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:21.308726 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:21.308893 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:21.309646 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:02:21.309658 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:02:21.309912 (+ 254us) client_negotiation.cc:769] Sending connection context
0809 13:02:21.310162 (+ 250us) client_negotiation.cc:240] Negotiation successful
0809 13:02:21.310873 (+ 711us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":179,"thread_start_us":384,"threads_started":1}
I20260809 13:02:21.315822 15850 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:02:21.317164 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.316638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44027 (local address 127.0.0.1:36658)
0809 13:02:21.316857 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.316996 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260809 13:02:21.317526 15795 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44027: connect: Connection refused (error 111)
I20260809 13:02:21.319358 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.318824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44027 (local address 127.0.0.1:36670)
0809 13:02:21.319004 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.319107 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 13:02:21.319584 15850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.125:44027: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44027: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.21.235.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.125.keytab.
Entry for principal kudu/127.21.235.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.125.keytab.
I20260809 13:02:21.362674 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.125:44027
--webserver_interface=127.21.235.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.125.keytab
--principal=kudu/127.21.235.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:46173,127.21.235.125:44027,127.21.235.124:41639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:02:21.369017 15850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } attempt: 1
I20260809 13:02:21.370316 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.369827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44027 (local address 127.0.0.1:34640)
0809 13:02:21.370049 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.370190 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 13:02:21.371315 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.370977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44027 (local address 127.0.0.1:34646)
0809 13:02:21.371117 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.371191 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 13:02:21.371495 15850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.125:44027: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44027: connect: Connection refused (error 111)
I20260809 13:02:21.439759 15850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } attempt: 2
I20260809 13:02:21.441233 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.440539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44027 (local address 127.0.0.1:34660)
0809 13:02:21.440862 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.441015 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 13:02:21.442435 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.441971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44027 (local address 127.0.0.1:34676)
0809 13:02:21.442146 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.442235 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 13:02:21.442641 15850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.125:44027: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44027: connect: Connection refused (error 111)
I20260809 13:02:21.533941 15850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } attempt: 3
I20260809 13:02:21.535399 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.534776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44027 (local address 127.0.0.1:34678)
0809 13:02:21.535051 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.535191 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260809 13:02:21.536628 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.536128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44027 (local address 127.0.0.1:34686)
0809 13:02:21.536311 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.536418 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 13:02:21.536871 15850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.125:44027: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44027: connect: Connection refused (error 111)
I20260809 13:02:21.680188 15850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } attempt: 4
I20260809 13:02:21.681632 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.680917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44027 (local address 127.0.0.1:34696)
0809 13:02:21.681317 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.681447 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193}
I20260809 13:02:21.682904 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.682462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44027 (local address 127.0.0.1:34712)
0809 13:02:21.682624 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.682728 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 13:02:21.683178 15850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.125:44027: Network error: Client connection negotiation failed: client connection to 127.21.235.125:44027: connect: Connection refused (error 111)
W20260809 13:02:21.712962 15857 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 13:02:21.713362 15857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:02:21.713466 15857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:02:21.713644 15857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:02:21.721235 15857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:02:21.721325 15857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:02:21.721414 15857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:02:21.721493 15857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:02:21.721585 15857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:02:21.731628 15857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:46173,127.21.235.125:44027,127.21.235.124:41639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.125.keytab
--principal=kudu/127.21.235.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.125:44027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.235.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:02:21.733000 15857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:02:21.734349 15857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:02:21.747267 15864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:21.747242 15866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:21.747331 15863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:02:21.748087 15857 server_base.cc:1034] running on GCE node
I20260809 13:02:21.748914 15857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:02:21.750490 15857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:02:21.751740 15857 hybrid_clock.cc:648] HybridClock initialized: now 1786280541751675 us; error 41 us; skew 500 ppm
I20260809 13:02:21.752084 15857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:02:21 dist-test-slave-gp6n krb5kdc[15756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280541, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:02:21.755587 15857 init.cc:377] Logged in from keytab as kudu/127.21.235.125@KRBTEST.COM (short username kudu)
I20260809 13:02:21.757095 15857 webserver.cc:460] Webserver started at http://127.21.235.125:37647/ using document root <none> and password file <none>
I20260809 13:02:21.757817 15857 fs_manager.cc:359] Metadata directory not provided
I20260809 13:02:21.757956 15857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:02:21.758291 15857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:02:21.761325 15857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-1/data/instance:
uuid: "8821f17613ed40a3851ccd32af0f24a2"
format_stamp: "Formatted at 2026-08-09 13:02:21 on dist-test-slave-gp6n"
server_key: "e2fbdf408baba2d10b070dd2f2f1e267"
server_key_iv: "db7e01864dd3b9cdcad48cb881a6a5c3"
server_key_version: "encryptionkey@0"
I20260809 13:02:21.762140 15857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-1/wal/instance:
uuid: "8821f17613ed40a3851ccd32af0f24a2"
format_stamp: "Formatted at 2026-08-09 13:02:21 on dist-test-slave-gp6n"
server_key: "e2fbdf408baba2d10b070dd2f2f1e267"
server_key_iv: "db7e01864dd3b9cdcad48cb881a6a5c3"
server_key_version: "encryptionkey@0"
I20260809 13:02:21.767483 15857 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 13:02:21.771129 15874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:02:21.772544 15857 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 13:02:21.772756 15857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-1/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-1/wal
uuid: "8821f17613ed40a3851ccd32af0f24a2"
format_stamp: "Formatted at 2026-08-09 13:02:21 on dist-test-slave-gp6n"
server_key: "e2fbdf408baba2d10b070dd2f2f1e267"
server_key_iv: "db7e01864dd3b9cdcad48cb881a6a5c3"
server_key_version: "encryptionkey@0"
I20260809 13:02:21.772989 15857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:02:21.786275 15857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:02:21.790954 15857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:02:21.791255 15857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:02:21.810482 15857 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.125:44027
I20260809 13:02:21.810523 15932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.125:44027 every 8 connection(s)
I20260809 13:02:21.811877 15857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-1/data/info.pb
I20260809 13:02:21.815833 15933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:02:21.816380 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 15857
I20260809 13:02:21.816584 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-1/wal/instance
I20260809 13:02:21.817169 22445 external_mini_cluster.cc:1428] Setting key c8d1f56aa18188fb212d27f8d8dbc84d
I20260809 13:02:21.825752 15933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:02:21 dist-test-slave-gp6n krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280541, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.125@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:02:21.849469 15935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.829120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51706 (local address 127.21.235.126:46173)
0809 13:02:21.829671 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:21.829679 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:21.830685 (+ 1006us) server_negotiation.cc:407] Connection header received
0809 13:02:21.832645 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:21.832653 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:21.832753 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:21.832905 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:21.835090 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:21.835809 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:21.838275 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:21.838761 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:21.838857 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:21.843298 (+ 4441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:21.843303 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:21.843306 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:21.843352 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:21.846085 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:21.846967 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:21.846973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:21.846975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:21.847109 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:21.847591 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:21.847595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:21.847597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:21.847888 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:21.848073 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:21.848822 (+ 749us) server_negotiation.cc:299] Negotiation successful
0809 13:02:21.849068 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":255,"threads_started":1}
I20260809 13:02:21.850533 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.829699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:46173 (local address 127.0.0.1:51706)
0809 13:02:21.830518 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.830565 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 13:02:21.832353 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:02:21.833166 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:02:21.833180 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:02:21.834003 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:02:21.834839 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:21.834866 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:21.835977 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:21.835986 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:21.838118 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:21.838135 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:21.839000 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:21.839009 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:21.839342 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:21.840149 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:02:21.840190 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:02:21.843056 (+ 2866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:02:21.846243 (+ 3187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:21.846252 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:21.846271 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:21.846824 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:21.847226 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:21.847231 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:21.847235 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:21.847454 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:21.848030 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:02:21.848040 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:02:21.848425 (+ 385us) client_negotiation.cc:769] Sending connection context
0809 13:02:21.848715 (+ 290us) client_negotiation.cc:240] Negotiation successful
0809 13:02:21.849256 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":338,"threads_started":1}
WARNING: no policy specified for kudu/127.21.235.124@KRBTEST.COM; defaulting to no policy
I20260809 13:02:21.852237 15933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.235.124@KRBTEST.COM" created.
Aug 09 13:02:21 dist-test-slave-gp6n krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280541, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.125@KRBTEST.COM for kudu/127.21.235.125@KRBTEST.COM
I20260809 13:02:21.866878 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.853230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44027 (local address 127.0.0.1:34718)
0809 13:02:21.853439 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.853457 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:02:21.853544 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:02:21.854581 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:02:21.854585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:02:21.855041 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:02:21.855172 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:21.855180 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:21.856556 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:21.856564 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:21.857386 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:21.857397 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:21.858204 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:21.858209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:21.858492 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:21.859130 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:02:21.859148 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:02:21.861115 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:02:21.864382 (+ 3267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:21.864387 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:21.864391 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:21.864750 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:21.865115 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:21.865121 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:21.865125 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:21.865254 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:21.865842 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:02:21.865848 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:02:21.865997 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 13:02:21.866150 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 13:02:21.866457 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":44}
I20260809 13:02:21.866961 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.853365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34718 (local address 127.21.235.125:44027)
0809 13:02:21.854108 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:21.854118 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:21.854148 (+ 30us) server_negotiation.cc:407] Connection header received
0809 13:02:21.854202 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:21.854208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:21.854383 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:21.854593 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:21.855344 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:21.856423 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:21.857534 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:21.858061 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:21.858195 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:21.861320 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:21.861331 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:21.861343 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:21.861370 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:21.864238 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:21.864890 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:21.864903 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:21.864905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:21.865013 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:21.865371 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:21.865378 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:21.865383 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:21.865724 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:21.865935 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:21.866300 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 13:02:21.866581 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":589,"thread_start_us":394,"threads_started":1}
I20260809 13:02:21.869964 15933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:02:21.871320 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.870700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:47854)
0809 13:02:21.870935 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.871085 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260809 13:02:21.871593 15879 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
I20260809 13:02:21.873328 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.872816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:47868)
0809 13:02:21.873032 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.873167 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 13:02:21.873569 15933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:41639: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.235.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.124.keytab.
Entry for principal kudu/127.21.235.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.124.keytab.
I20260809 13:02:21.895788 22445 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
/tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.124:41639
--webserver_interface=127.21.235.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.124.keytab
--principal=kudu/127.21.235.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.235.84:43295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.235.126:46173,127.21.235.125:44027,127.21.235.124:41639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:02:21.922909 15933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } attempt: 1
I20260809 13:02:21.924435 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.923731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:47870)
0809 13:02:21.923959 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.924067 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 13:02:21.925598 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.925214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:47872)
0809 13:02:21.925384 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.925473 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 13:02:21.925819 15933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:41639: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
I20260809 13:02:21.982527 15850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } attempt: 5
Aug 09 13:02:21 dist-test-slave-gp6n krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280541, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for kudu/127.21.235.125@KRBTEST.COM
I20260809 13:02:21.994045 15933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } attempt: 2
I20260809 13:02:21.995154 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.994699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:47886)
0809 13:02:21.994867 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.994961 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 13:02:21.996256 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.995790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:47902)
0809 13:02:21.995959 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.996075 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 13:02:21.996649 15933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:41639: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
I20260809 13:02:21.996866 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.983297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44027 (local address 127.0.0.1:34732)
0809 13:02:21.983512 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.983535 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 13:02:21.983671 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:02:21.984045 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:02:21.984051 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:02:21.984466 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:02:21.984670 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:21.984679 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:21.985698 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:21.985705 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:21.986636 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:21.986648 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:21.987395 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:21.987400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:21.987619 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:21.988250 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:02:21.988268 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:02:21.990613 (+ 2345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:02:21.993684 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:21.993689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:21.993693 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:21.994098 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:21.994597 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:21.994604 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:21.994611 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:21.994802 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:21.995719 (+ 917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:02:21.995723 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:02:21.995911 (+ 188us) client_negotiation.cc:769] Sending connection context
0809 13:02:21.996106 (+ 195us) client_negotiation.cc:240] Negotiation successful
0809 13:02:21.996453 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 13:02:21.996873 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.983469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34732 (local address 127.21.235.125:44027)
0809 13:02:21.983660 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:21.983666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:21.983690 (+ 24us) server_negotiation.cc:407] Connection header received
0809 13:02:21.983790 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:21.983796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:21.983886 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:21.983997 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:21.984827 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:21.985552 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:21.986799 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:21.987231 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:21.987386 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:21.990800 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:21.990805 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:21.990808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:21.990841 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:21.993544 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:21.994224 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:21.994230 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:21.994237 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:21.994451 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:21.994914 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:21.994921 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:21.994924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:21.995513 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:21.995663 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:21.996272 (+ 609us) server_negotiation.cc:299] Negotiation successful
0809 13:02:21.996526 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 13:02:21.997931 15850 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:02:21.999122 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.998660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:47906)
0809 13:02:21.998844 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:21.998941 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 13:02:22.000218 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:21.999779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:47912)
0809 13:02:21.999977 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:22.000089 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 13:02:22.000487 15850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:41639: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
I20260809 13:02:22.051766 15850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } attempt: 1
I20260809 13:02:22.053223 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.052600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:47926)
0809 13:02:22.052884 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:22.053009 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260809 13:02:22.054756 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.054229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:47940)
0809 13:02:22.054464 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:22.054576 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 13:02:22.055049 15850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:41639: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
I20260809 13:02:22.087917 15933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } attempt: 3
I20260809 13:02:22.089118 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.088636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:47956)
0809 13:02:22.088840 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:22.088957 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 13:02:22.090205 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.089790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:47962)
0809 13:02:22.090023 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:22.090092 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260809 13:02:22.090461 15933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:41639: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
I20260809 13:02:22.123323 15850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } attempt: 2
I20260809 13:02:22.124889 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.124126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:47970)
0809 13:02:22.124446 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:22.124595 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260809 13:02:22.126103 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.125717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:47986)
0809 13:02:22.125920 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:22.125998 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260809 13:02:22.126443 15850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:41639: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
I20260809 13:02:22.232676 15850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } attempt: 3
I20260809 13:02:22.233727 15933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } attempt: 4
I20260809 13:02:22.234032 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.233390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:47994)
0809 13:02:22.233676 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:22.233814 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 13:02:22.234938 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.234390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:48006)
0809 13:02:22.234655 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:22.234754 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 13:02:22.235256 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.234835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:48018)
0809 13:02:22.235040 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:22.235126 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260809 13:02:22.235452 15850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:41639: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
I20260809 13:02:22.236076 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.235672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:48034)
0809 13:02:22.235846 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:22.235938 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 13:02:22.236299 15933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.235.124:41639: Network error: Client connection negotiation failed: client connection to 127.21.235.124:41639: connect: Connection refused (error 111)
W20260809 13:02:22.263473 15943 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 13:02:22.263785 15943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:02:22.263857 15943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:02:22.263942 15943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:02:22.271251 15943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:02:22.271337 15943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:02:22.271395 15943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:02:22.271438 15943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:02:22.271500 15943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:02:22.281385 15943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.235.84:43295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.235.126:46173,127.21.235.125:44027,127.21.235.124:41639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/krb5kdc/kudu_127.21.235.124.keytab
--principal=kudu/127.21.235.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.235.124:41639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.235.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:02:22.282737 15943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:02:22.284037 15943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:02:22.296330 15949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:22.296429 15950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:02:22.296406 15952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:02:22.297659 15943 server_base.cc:1034] running on GCE node
I20260809 13:02:22.298578 15943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:02:22.299945 15943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:02:22.301189 15943 hybrid_clock.cc:648] HybridClock initialized: now 1786280542301110 us; error 57 us; skew 500 ppm
I20260809 13:02:22.301508 15943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:02:22 dist-test-slave-gp6n krb5kdc[15756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280542, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:02:22.304816 15943 init.cc:377] Logged in from keytab as kudu/127.21.235.124@KRBTEST.COM (short username kudu)
I20260809 13:02:22.306324 15943 webserver.cc:460] Webserver started at http://127.21.235.124:45587/ using document root <none> and password file <none>
I20260809 13:02:22.307072 15943 fs_manager.cc:359] Metadata directory not provided
I20260809 13:02:22.307159 15943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:02:22.307430 15943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:02:22.310453 15943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-2/data/instance:
uuid: "5022831b0878469f92478e56911a6090"
format_stamp: "Formatted at 2026-08-09 13:02:22 on dist-test-slave-gp6n"
server_key: "cdd95ee64d442e7a2e4f126436c072b7"
server_key_iv: "95653a71c75f30e1c99a26dd9ba3f687"
server_key_version: "encryptionkey@0"
I20260809 13:02:22.311221 15943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-2/wal/instance:
uuid: "5022831b0878469f92478e56911a6090"
format_stamp: "Formatted at 2026-08-09 13:02:22 on dist-test-slave-gp6n"
server_key: "cdd95ee64d442e7a2e4f126436c072b7"
server_key_iv: "95653a71c75f30e1c99a26dd9ba3f687"
server_key_version: "encryptionkey@0"
I20260809 13:02:22.316392 15943 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 13:02:22.320096 15959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:02:22.321415 15943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:02:22.321615 15943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-2/data,/tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-2/wal
uuid: "5022831b0878469f92478e56911a6090"
format_stamp: "Formatted at 2026-08-09 13:02:22 on dist-test-slave-gp6n"
server_key: "cdd95ee64d442e7a2e4f126436c072b7"
server_key_iv: "95653a71c75f30e1c99a26dd9ba3f687"
server_key_version: "encryptionkey@0"
I20260809 13:02:22.321852 15943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:02:22.339687 15943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:02:22.344084 15943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:02:22.344398 15943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:02:22.362921 15943 rpc_server.cc:307] RPC server started. Bound to: 127.21.235.124:41639
I20260809 13:02:22.362963 16015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.235.124:41639 every 8 connection(s)
I20260809 13:02:22.364272 15943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-2/data/info.pb
I20260809 13:02:22.368096 16016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:02:22.369612 22445 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu as pid 15943
I20260809 13:02:22.369809 22445 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjg7NUQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280163688078-22445-0/minicluster-data/master-2/wal/instance
I20260809 13:02:22.370357 22445 external_mini_cluster.cc:1428] Setting key e7f374cc676e04500465384e1cea589d
I20260809 13:02:22.378019 16016 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:02:22 dist-test-slave-gp6n krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.124@KRBTEST.COM
Aug 09 13:02:22 dist-test-slave-gp6n krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280542, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.124@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:02:22.394152 16019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.372292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48050 (local address 127.21.235.124:41639)
0809 13:02:22.373095 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:22.373106 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:22.373147 (+ 41us) server_negotiation.cc:407] Connection header received
0809 13:02:22.374846 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:22.374868 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:22.375277 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:22.375620 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:22.377433 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.378997 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.380685 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.381977 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.382150 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:22.386779 (+ 4629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:22.386786 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:22.386796 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:22.386831 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:22.390149 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.390919 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.390928 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.390943 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.391088 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.391479 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.391486 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.391489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.391991 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:22.392318 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:22.392745 (+ 427us) server_negotiation.cc:299] Negotiation successful
0809 13:02:22.393160 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":381,"threads_started":1}
I20260809 13:02:22.396294 16021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.380763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51710 (local address 127.21.235.126:46173)
0809 13:02:22.381699 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:22.381707 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:22.382375 (+ 668us) server_negotiation.cc:407] Connection header received
0809 13:02:22.382485 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:22.382488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:22.382557 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:22.382642 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:22.384148 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.384842 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.386308 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.386806 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.386954 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:22.390634 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:22.390639 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:22.390643 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:22.390683 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:22.393465 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.394143 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.394150 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.394156 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.394352 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.394807 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.394814 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.394817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.395092 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:22.395239 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:22.395681 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 13:02:22.395899 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":663,"threads_started":1}
I20260809 13:02:22.396345 16020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.380693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.126:46173 (local address 127.0.0.1:51710)
0809 13:02:22.382013 (+ 1320us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:22.382075 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 13:02:22.382241 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:02:22.382749 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:02:22.382759 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:02:22.383630 (+ 871us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:02:22.383977 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:22.383991 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.385002 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.385008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:22.386150 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:22.386164 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.387001 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.387009 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:22.387199 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:22.387886 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:02:22.387903 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:02:22.390380 (+ 2477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:02:22.393591 (+ 3211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:22.393604 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:22.393611 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:22.394021 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:22.394527 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:22.394532 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:22.394535 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:22.394689 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:22.395229 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:02:22.395237 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:02:22.395396 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 13:02:22.395573 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 13:02:22.395899 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1129,"thread_start_us":986,"threads_started":1}
I20260809 13:02:22.397720 16016 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:02:22 dist-test-slave-gp6n krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280542, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.124@KRBTEST.COM for kudu/127.21.235.125@KRBTEST.COM
Aug 09 13:02:22 dist-test-slave-gp6n krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.126@KRBTEST.COM
I20260809 13:02:22.410708 16020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.398527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.125:44027 (local address 127.0.0.1:34738)
0809 13:02:22.398707 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:22.398724 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:02:22.398854 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:02:22.399178 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:02:22.399182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:02:22.399544 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:02:22.399662 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:22.399670 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.400625 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.400632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:22.401485 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:22.401496 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.402186 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.402191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:22.402390 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:22.402940 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:02:22.402955 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:02:22.405140 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:02:22.408115 (+ 2975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:22.408120 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:22.408124 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:22.408476 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:22.408853 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:22.408858 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:22.408861 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:22.409010 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:22.409660 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:02:22.409666 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:02:22.409837 (+ 171us) client_negotiation.cc:769] Sending connection context
0809 13:02:22.410023 (+ 186us) client_negotiation.cc:240] Negotiation successful
0809 13:02:22.410302 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 13:02:22.410750 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.398517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34738 (local address 127.21.235.125:44027)
0809 13:02:22.398681 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:22.398686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:22.398808 (+ 122us) server_negotiation.cc:407] Connection header received
0809 13:02:22.398969 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:22.398974 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:22.399058 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:22.399160 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:22.399825 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.400494 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.401624 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.402042 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.402176 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:22.405320 (+ 3144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:22.405325 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:22.405328 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:22.405358 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:22.407976 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.408592 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.408599 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.408602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.408740 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.409119 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.409124 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.409128 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.409406 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:22.409709 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:22.410139 (+ 430us) server_negotiation.cc:299] Negotiation successful
0809 13:02:22.410355 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 13:02:22.411280 16021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.397140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51718 (local address 127.21.235.126:46173)
0809 13:02:22.397289 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:22.397295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:22.397398 (+ 103us) server_negotiation.cc:407] Connection header received
0809 13:02:22.397592 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:22.397597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:22.397672 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:22.397774 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:22.399206 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.399961 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.401160 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.401584 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.401720 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:22.405872 (+ 4152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:22.405881 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:22.405886 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:22.405926 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:22.408691 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.409386 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.409391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.409393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.409509 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.409875 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.409881 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.409884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.410147 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:22.410470 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:22.410765 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 13:02:22.410990 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 13:02:22.412532 16016 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:02:22.412652 15850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } attempt: 4
Aug 09 13:02:22 dist-test-slave-gp6n krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280542, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.124@KRBTEST.COM for kudu/127.21.235.124@KRBTEST.COM
Aug 09 13:02:22 dist-test-slave-gp6n krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.235.125@KRBTEST.COM
Aug 09 13:02:22 dist-test-slave-gp6n krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280541, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.126@KRBTEST.COM for kudu/127.21.235.124@KRBTEST.COM
I20260809 13:02:22.428658 16020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.413210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:48074)
0809 13:02:22.413415 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:22.413433 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:02:22.413530 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:02:22.414232 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:02:22.414241 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:02:22.414765 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:02:22.415086 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:22.415095 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.416545 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.416551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:22.417890 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:22.417905 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.419088 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.419091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:22.419337 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:22.419871 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:02:22.419886 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:02:22.421831 (+ 1945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:02:22.425491 (+ 3660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:22.425494 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:22.425496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:22.426114 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:22.426624 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:22.426631 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:22.426637 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:22.426772 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:22.427526 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:02:22.427536 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:02:22.427686 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 13:02:22.427859 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 13:02:22.428201 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":54,"spinlock_wait_cycles":29952}
I20260809 13:02:22.428928 16019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.413656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48074 (local address 127.21.235.124:41639)
0809 13:02:22.413818 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:22.413823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:22.413839 (+ 16us) server_negotiation.cc:407] Connection header received
0809 13:02:22.413899 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:22.413906 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:22.413976 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:22.414058 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:22.415212 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.416425 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.418052 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.418820 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.418944 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:22.422007 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:22.422012 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:22.422015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:22.422041 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:22.425342 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.426262 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.426267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.426271 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.426481 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.426892 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.426906 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.426911 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.427312 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:22.427457 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:22.428041 (+ 584us) server_negotiation.cc:299] Negotiation successful
0809 13:02:22.428455 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":62}
I20260809 13:02:22.431609 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.413197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:48064)
0809 13:02:22.413669 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:22.413687 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:02:22.413781 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:02:22.414829 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:02:22.414857 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:02:22.415296 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:02:22.415481 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:22.415493 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.416989 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.416994 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:22.418433 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:22.418447 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.420113 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.420120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:22.420338 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:22.420979 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:02:22.421005 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:02:22.424325 (+ 3320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:02:22.428616 (+ 4291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:22.428623 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:22.428628 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:22.429142 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:22.429563 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:22.429568 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:22.429572 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:22.429717 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:22.430389 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:02:22.430395 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:02:22.430657 (+ 262us) client_negotiation.cc:769] Sending connection context
0809 13:02:22.430864 (+ 207us) client_negotiation.cc:240] Negotiation successful
0809 13:02:22.431174 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348}
I20260809 13:02:22.431694 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.413740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48064 (local address 127.21.235.124:41639)
0809 13:02:22.414509 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:22.414515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:22.414538 (+ 23us) server_negotiation.cc:407] Connection header received
0809 13:02:22.414586 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:22.414591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:22.414654 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:22.414764 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:22.415671 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.416828 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.418597 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.419898 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.423819 (+ 3921us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:22.424555 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:22.424563 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:22.424569 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:22.424604 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:22.428463 (+ 3859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.429318 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.429321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.429324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.429444 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.429872 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.429879 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.429883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.430217 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:22.430449 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:22.430953 (+ 504us) server_negotiation.cc:299] Negotiation successful
0809 13:02:22.431173 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":618,"thread_start_us":432,"threads_started":1}
I20260809 13:02:22.433063 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.413275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34744 (local address 127.21.235.125:44027)
0809 13:02:22.413433 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:22.413439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:22.413458 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:02:22.413507 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:22.413512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:22.413838 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:22.413941 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:22.415610 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.416781 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.417960 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.418599 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.418737 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:22.425154 (+ 6417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:22.425168 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:22.425173 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:22.425207 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:22.429042 (+ 3835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.430017 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.430024 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.430028 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.430194 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.430685 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.430690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.430693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.431054 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:22.431349 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:22.432482 (+ 1133us) server_negotiation.cc:299] Negotiation successful
0809 13:02:22.432724 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 13:02:22.435165 16016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090: Bootstrap starting.
I20260809 13:02:22.436671 15850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f: Bootstrap starting.
I20260809 13:02:22.438678 16016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090: Neither blocks nor log segments found. Creating new log.
I20260809 13:02:22.439910 16016 log.cc:824] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090: Log is configured to *not* fsync() on all Append() calls
I20260809 13:02:22.440703 15850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f: Neither blocks nor log segments found. Creating new log.
I20260809 13:02:22.442083 15850 log.cc:824] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f: Log is configured to *not* fsync() on all Append() calls
I20260809 13:02:22.442369 22445 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:02:22.443826 16016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090: No bootstrap required, opened a new log
I20260809 13:02:22.446547 15850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f: No bootstrap required, opened a new log
I20260809 13:02:22.445537 15823 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "602c8155050244319e250864f6aaa80f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51718
I20260809 13:02:22.451135 16016 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602c8155050244319e250864f6aaa80f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } } peers { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } } peers { permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } }
I20260809 13:02:22.451488 16016 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5022831b0878469f92478e56911a6090, State: Initialized, Role: FOLLOWER
I20260809 13:02:22.451922 16016 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602c8155050244319e250864f6aaa80f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } } peers { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } } peers { permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } }
I20260809 13:02:22.452826 16026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602c8155050244319e250864f6aaa80f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } } peers { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } } peers { permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } } }
I20260809 13:02:22.453258 16026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:02:22.453985 16016 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:02:22.455952 15850 raft_consensus.cc:348] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602c8155050244319e250864f6aaa80f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } } peers { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } } peers { permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } }
I20260809 13:02:22.456420 15850 raft_consensus.cc:729] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602c8155050244319e250864f6aaa80f, State: Initialized, Role: FOLLOWER
I20260809 13:02:22.456974 15850 consensus_queue.cc:260] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602c8155050244319e250864f6aaa80f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } } peers { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } } peers { permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } }
I20260809 13:02:22.458246 16033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602c8155050244319e250864f6aaa80f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } } peers { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } } peers { permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } } }
I20260809 13:02:22.458714 16033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:02:22.459849 15850 sys_catalog.cc:564] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:02:22.459935 15823 raft_consensus.cc:482] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 13:02:22.460172 15823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:02:22.464131 15823 raft_consensus.cc:504] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602c8155050244319e250864f6aaa80f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } } peers { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } } peers { permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } }
I20260809 13:02:22.466109 15823 leader_election.cc:288] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [CANDIDATE]: Term 1 election: Requested vote from peers 8821f17613ed40a3851ccd32af0f24a2 (127.21.235.125:44027), 5022831b0878469f92478e56911a6090 (127.21.235.124:41639)
I20260809 13:02:22.467314 15989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "602c8155050244319e250864f6aaa80f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5022831b0878469f92478e56911a6090"
I20260809 13:02:22.467883 15989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:02:22.472035 15989 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602c8155050244319e250864f6aaa80f in term 1.
I20260809 13:02:22.472769 15795 leader_election.cc:302] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5022831b0878469f92478e56911a6090, 602c8155050244319e250864f6aaa80f; no voters:
I20260809 13:02:22.473325 16033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:02:22.463590 15943 master_runner.cc:186] Error getting master registration for 127.21.235.126:46173: OK, instance_id { permanent_uuid: "602c8155050244319e250864f6aaa80f" instance_seqno: 1786280541270953 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 13:02:22.477984 15943 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 13:02:22.478456 16033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [term 1 LEADER]: Becoming Leader. State: Replica: 602c8155050244319e250864f6aaa80f, State: Running, Role: LEADER
I20260809 13:02:22.479223 16033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602c8155050244319e250864f6aaa80f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } } peers { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } } peers { permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } }
W20260809 13:02:22.481757 16039 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 13:02:22.481904 16039 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 13:02:22.484831 16042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 602c8155050244319e250864f6aaa80f. Latest consensus state: current_term: 1 leader_uuid: "602c8155050244319e250864f6aaa80f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602c8155050244319e250864f6aaa80f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } } peers { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } } peers { permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } } }
I20260809 13:02:22.485363 16042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:22.486742 16050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260809 13:02:22.509908 15797 tablet.cc:2369] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:02:22.512319 16050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:02:22.536545 15933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } attempt: 5
Aug 09 13:02:22 dist-test-slave-gp6n krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280541, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.235.125@KRBTEST.COM for kudu/127.21.235.124@KRBTEST.COM
I20260809 13:02:22.552487 16021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.488132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51722 (local address 127.21.235.126:46173)
0809 13:02:22.488326 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:22.488334 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:22.510315 (+ 21981us) server_negotiation.cc:407] Connection header received
0809 13:02:22.513591 (+ 3276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:22.513599 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:22.513698 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:22.513785 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:22.524704 (+ 10919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.526141 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.535456 (+ 9315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.536199 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.536314 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:22.540465 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:22.540476 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:22.540484 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:22.540562 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:22.545526 (+ 4964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.547447 (+ 1921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.547454 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.547457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.547662 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.550324 (+ 2662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.550330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.550333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.550808 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:22.551114 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:22.551761 (+ 647us) server_negotiation.cc:299] Negotiation successful
0809 13:02:22.552024 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 13:02:22.560510 15989 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090 [term 1 FOLLOWER]: Refusing update from remote peer 602c8155050244319e250864f6aaa80f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:02:22.562042 16042 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:02:22.562717 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.537406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.235.124:41639 (local address 127.0.0.1:48088)
0809 13:02:22.537850 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 13:02:22.537876 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 13:02:22.538053 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:02:22.538619 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:02:22.538627 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:02:22.539237 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:02:22.539483 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:22.539499 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.541075 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.541081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:22.542702 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:02:22.542719 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.544157 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.544165 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:02:22.544402 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:22.545069 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:02:22.545098 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:02:22.548436 (+ 3338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:02:22.559313 (+ 10877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:22.559318 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:22.559322 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:22.559962 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:22.560510 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:02:22.560515 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:02:22.560520 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:02:22.560770 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:02:22.561563 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:02:22.561570 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:02:22.561769 (+ 199us) client_negotiation.cc:769] Sending connection context
0809 13:02:22.561918 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 13:02:22.562240 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282}
I20260809 13:02:22.563797 16019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.537453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48088 (local address 127.21.235.124:41639)
0809 13:02:22.537672 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:22.537681 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:22.537974 (+ 293us) server_negotiation.cc:407] Connection header received
0809 13:02:22.538173 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:22.538179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:22.538272 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:22.538392 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:22.539694 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.540872 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.542934 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.543717 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.543876 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:22.548692 (+ 4816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:22.548702 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:22.548709 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:22.548756 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:22.559134 (+ 10378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.560160 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.560167 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.560172 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.560347 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.560977 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.560985 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.560988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.561397 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:22.563003 (+ 1606us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:22.563154 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 13:02:22.563411 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 13:02:22.568238 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.511111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48086 (local address 127.21.235.124:41639)
0809 13:02:22.511380 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:22.511388 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:22.518887 (+ 7499us) server_negotiation.cc:407] Connection header received
0809 13:02:22.522101 (+ 3214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:22.522110 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:22.522204 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:22.522280 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:22.531779 (+ 9499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.533025 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.535131 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.536115 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.536294 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:22.550101 (+ 13807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:22.550110 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:22.550118 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:22.550174 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:22.556383 (+ 6209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.563209 (+ 6826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.563215 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.563218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.563391 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.564924 (+ 1533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.564930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.564934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.565320 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:22.565647 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:22.567566 (+ 1919us) server_negotiation.cc:299] Negotiation successful
0809 13:02:22.567824 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 13:02:22.570797 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.494833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34756 (local address 127.21.235.125:44027)
0809 13:02:22.495045 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:22.495052 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:22.518067 (+ 23015us) server_negotiation.cc:407] Connection header received
0809 13:02:22.519199 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:22.519206 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:22.519307 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:22.519426 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:22.528451 (+ 9025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.529716 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.540417 (+ 10701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:22.541215 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:22.541319 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:22.557699 (+ 16380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:02:22.557709 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:02:22.557716 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:02:22.557764 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:02:22.561843 (+ 4079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.563894 (+ 2051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.563902 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.563908 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.564074 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:02:22.566166 (+ 2092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:02:22.566173 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:02:22.566177 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:02:22.566676 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:02:22.566967 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:22.569926 (+ 2959us) server_negotiation.cc:299] Negotiation successful
0809 13:02:22.570205 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":78}
I20260809 13:02:22.580008 15933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2: Bootstrap starting.
W20260809 13:02:22.596443 15965 tablet.cc:2369] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:02:22.600477 15933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2: Neither blocks nor log segments found. Creating new log.
I20260809 13:02:22.609133 15933 log.cc:824] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2: Log is configured to *not* fsync() on all Append() calls
I20260809 13:02:22.611989 16026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 602c8155050244319e250864f6aaa80f. Latest consensus state: current_term: 1 leader_uuid: "602c8155050244319e250864f6aaa80f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602c8155050244319e250864f6aaa80f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } } peers { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } } peers { permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } } }
I20260809 13:02:22.612393 16026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:02:22.621487 16060 mvcc.cc:205] Tried to move back new op lower bound from 7316605102303141888 to 7316605102003593216. Current Snapshot: MvccSnapshot[applied={T|T < 7316605102303141888}]
I20260809 13:02:22.622898 16042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "602c8155050244319e250864f6aaa80f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602c8155050244319e250864f6aaa80f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } } peers { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } } peers { permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } } }
I20260809 13:02:22.623936 16042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:22.626698 15933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2: No bootstrap required, opened a new log
I20260809 13:02:22.630277 16026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "602c8155050244319e250864f6aaa80f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602c8155050244319e250864f6aaa80f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } } peers { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } } peers { permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } } }
I20260809 13:02:22.630558 16026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:02:22.635885 16050 catalog_manager.cc:1347] Generated new cluster ID: 233e41cd58c746f9a16243f63b60733f
I20260809 13:02:22.636003 16050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:02:22.637997 15933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602c8155050244319e250864f6aaa80f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } } peers { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } } peers { permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } }
I20260809 13:02:22.638427 15933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8821f17613ed40a3851ccd32af0f24a2, State: Initialized, Role: FOLLOWER
I20260809 13:02:22.638914 15933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602c8155050244319e250864f6aaa80f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } } peers { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } } peers { permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } }
I20260809 13:02:22.631338 15774 master_runner.cc:186] Error getting master registration for 127.21.235.125:44027: OK, instance_id { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" instance_seqno: 1786280541804991 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 13:02:22.673374 15774 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 13:02:22.677768 15933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:02:22.677793 15904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "602c8155050244319e250864f6aaa80f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8821f17613ed40a3851ccd32af0f24a2"
I20260809 13:02:22.678277 15903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:02:22.682711 15903 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2 [term 1 FOLLOWER]: Refusing update from remote peer 602c8155050244319e250864f6aaa80f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:02:22.687014 16042 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:02:22.677414 16073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602c8155050244319e250864f6aaa80f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } } peers { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } } peers { permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } } }
I20260809 13:02:22.687790 16073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:02:22.690059 15904 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602c8155050244319e250864f6aaa80f in term 1.
I20260809 13:02:22.731464 16087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "602c8155050244319e250864f6aaa80f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602c8155050244319e250864f6aaa80f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } } peers { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } } peers { permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } } }
I20260809 13:02:22.731863 16087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:02:22.732216 16073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 602c8155050244319e250864f6aaa80f. Latest consensus state: current_term: 1 leader_uuid: "602c8155050244319e250864f6aaa80f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602c8155050244319e250864f6aaa80f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } } peers { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } } peers { permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } } }
I20260809 13:02:22.732547 16073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:02:22.766124 16042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "602c8155050244319e250864f6aaa80f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602c8155050244319e250864f6aaa80f" member_type: VOTER last_known_addr { host: "127.21.235.126" port: 46173 } } peers { permanent_uuid: "8821f17613ed40a3851ccd32af0f24a2" member_type: VOTER last_known_addr { host: "127.21.235.125" port: 44027 } } peers { permanent_uuid: "5022831b0878469f92478e56911a6090" member_type: VOTER last_known_addr { host: "127.21.235.124" port: 41639 } } }
I20260809 13:02:22.770376 16042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f [sys.catalog]: This master's current role is: LEADER
I20260809 13:02:22.770819 16050 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:02:22.773449 16050 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:02:22.813272 16050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 602c8155050244319e250864f6aaa80f: Generated new TSK 0
I20260809 13:02:22.814477 16050 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:02:22.822583 16094 catalog_manager.cc:1259] Loaded cluster ID: 233e41cd58c746f9a16243f63b60733f
I20260809 13:02:22.822780 16094 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2: loading cluster ID for follower catalog manager: success
I20260809 13:02:22.839335 16094 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2: acquiring CA information for follower catalog manager: success
I20260809 13:02:22.842165 16094 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8821f17613ed40a3851ccd32af0f24a2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 13:02:22.958608 16021 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:22.946460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51728 (local address 127.21.235.126:46173)
0809 13:02:22.946708 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:22.946716 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:22.954306 (+ 7590us) server_negotiation.cc:407] Connection header received
0809 13:02:22.956647 (+ 2341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:22.956654 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:22.956749 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:22.956844 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:22.958279 (+ 1435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51728: BlockingRecv error: recv got EOF from 127.0.0.1:51728 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260809 13:02:22.962601 15939 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:22.949796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34766 (local address 127.21.235.125:44027)
0809 13:02:22.950053 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:22.950061 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:22.959934 (+ 9873us) server_negotiation.cc:407] Connection header received
0809 13:02:22.960931 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:22.960938 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:22.961029 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:22.961113 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:22.962289 (+ 1176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34766: BlockingRecv error: recv got EOF from 127.0.0.1:34766 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 13:02:22.965934 16022 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:22.954998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48098 (local address 127.21.235.124:41639)
0809 13:02:22.955349 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:22.955365 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:22.963546 (+ 8181us) server_negotiation.cc:407] Connection header received
0809 13:02:22.964293 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:22.964301 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:22.964388 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:22.964481 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:22.965669 (+ 1188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48098: BlockingRecv error: recv got EOF from 127.0.0.1:48098 (error 108)
Metrics: {"server-negotiator.queue_time_us":201}
I20260809 13:02:23.019137 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:22.999633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34778 (local address 127.21.235.125:44027)
0809 13:02:22.999828 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:22.999834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.011049 (+ 11215us) server_negotiation.cc:407] Connection header received
0809 13:02:23.011357 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.011364 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.011625 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.011784 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:23.012557 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.013706 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.015861 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.016579 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.016686 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:23.017511 (+ 825us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:23.018242 (+ 731us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:23.018368 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:23.018645 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 13:02:23.018747 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 13:02:23.062026 16022 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:23.052403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48100 (local address 127.21.235.124:41639)
0809 13:02:23.052632 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.052638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.059631 (+ 6993us) server_negotiation.cc:407] Connection header received
0809 13:02:23.060244 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.060252 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.060392 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.060491 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:23.061768 (+ 1277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48100: BlockingRecv error: recv got EOF from 127.0.0.1:48100 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 13:02:23.110296 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:23.095707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34790 (local address 127.21.235.125:44027)
0809 13:02:23.095927 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.095934 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.103038 (+ 7104us) server_negotiation.cc:407] Connection header received
0809 13:02:23.103579 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.103586 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.103688 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.103756 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:23.104500 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.105667 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.107849 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.108556 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.108657 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:23.109380 (+ 723us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:23.109576 (+ 196us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:23.109643 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:23.109884 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 13:02:23.109978 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260809 13:02:23.163166 16022 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:23.153282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48104 (local address 127.21.235.124:41639)
0809 13:02:23.153539 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.153547 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.161075 (+ 7528us) server_negotiation.cc:407] Connection header received
0809 13:02:23.161444 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.161453 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.161563 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.161640 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:23.162870 (+ 1230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48104: BlockingRecv error: recv got EOF from 127.0.0.1:48104 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 13:02:23.218396 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:23.203226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34804 (local address 127.21.235.125:44027)
0809 13:02:23.203444 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.203453 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.211441 (+ 7988us) server_negotiation.cc:407] Connection header received
0809 13:02:23.211952 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.211964 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.212056 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.212126 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:23.212839 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.214084 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.216031 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.216646 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.216755 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:23.217405 (+ 650us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:23.217564 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:23.217633 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:23.217903 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 13:02:23.217993 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260809 13:02:23.230335 16022 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:23.227469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48106 (local address 127.21.235.124:41639)
0809 13:02:23.227731 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.227737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.228459 (+ 722us) server_negotiation.cc:407] Connection header received
0809 13:02:23.228886 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.228895 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.229005 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.229081 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:23.230101 (+ 1020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48106: BlockingRecv error: recv got EOF from 127.0.0.1:48106 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 13:02:23.262075 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:23.252947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34818 (local address 127.21.235.125:44027)
0809 13:02:23.253188 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.253196 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.254964 (+ 1768us) server_negotiation.cc:407] Connection header received
0809 13:02:23.255362 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.255369 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.255492 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.255563 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:23.256252 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.257538 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.259618 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.260455 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.260556 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:23.261214 (+ 658us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:23.261358 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:23.261427 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:23.261656 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 13:02:23.261751 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
W20260809 13:02:23.276554 16022 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:23.269564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48122 (local address 127.21.235.124:41639)
0809 13:02:23.269811 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.269817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.274876 (+ 5059us) server_negotiation.cc:407] Connection header received
0809 13:02:23.275089 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.275098 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.275213 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.275316 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:23.276278 (+ 962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48122: BlockingRecv error: recv got EOF from 127.0.0.1:48122 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
I20260809 13:02:23.301527 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:23.295035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34822 (local address 127.21.235.125:44027)
0809 13:02:23.295201 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.295207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.295226 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:02:23.295445 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.295457 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.295548 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.295624 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:23.296334 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.297489 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.299516 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.300107 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.300248 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:23.300754 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:23.300860 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:23.300934 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:23.301099 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 13:02:23.301163 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 13:02:23.309690 16022 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:23.307612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48136 (local address 127.21.235.124:41639)
0809 13:02:23.307866 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.307872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.308188 (+ 316us) server_negotiation.cc:407] Connection header received
0809 13:02:23.308421 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.308426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.308498 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.308584 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:23.309474 (+ 890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48136: BlockingRecv error: recv got EOF from 127.0.0.1:48136 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 13:02:23.336148 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:23.329866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34832 (local address 127.21.235.125:44027)
0809 13:02:23.330079 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.330086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.330457 (+ 371us) server_negotiation.cc:407] Connection header received
0809 13:02:23.330675 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.330681 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.330795 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.330911 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:23.331354 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.332576 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.333990 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.334732 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.334870 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:23.335316 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:23.335474 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:23.335583 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:23.335745 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 13:02:23.335843 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 13:02:23.345269 16022 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:23.343102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48138 (local address 127.21.235.124:41639)
0809 13:02:23.343373 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.343383 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.343672 (+ 289us) server_negotiation.cc:407] Connection header received
0809 13:02:23.343850 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.343855 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.343934 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.344036 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:23.345022 (+ 986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48138: BlockingRecv error: recv got EOF from 127.0.0.1:48138 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 13:02:23.391739 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:23.385090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34840 (local address 127.21.235.125:44027)
0809 13:02:23.385338 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.385345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.385925 (+ 580us) server_negotiation.cc:407] Connection header received
0809 13:02:23.386147 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.386152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.386237 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.386373 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:23.386861 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.387705 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.389685 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.390218 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.390358 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:23.390870 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:23.391014 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:23.391094 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:23.391290 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 13:02:23.391381 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260809 13:02:23.402599 16022 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:23.400252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48142 (local address 127.21.235.124:41639)
0809 13:02:23.400518 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.400526 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.400886 (+ 360us) server_negotiation.cc:407] Connection header received
0809 13:02:23.401076 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.401081 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.401157 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.401246 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:23.402268 (+ 1022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48142: BlockingRecv error: recv got EOF from 127.0.0.1:48142 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 13:02:23.475438 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:23.469932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34854 (local address 127.21.235.125:44027)
0809 13:02:23.470134 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.470140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.470561 (+ 421us) server_negotiation.cc:407] Connection header received
0809 13:02:23.470785 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.470790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.470881 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.470978 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:23.471470 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.472349 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.473698 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.474185 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.474293 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:23.474760 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:23.474898 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:23.474992 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:23.475111 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 13:02:23.475184 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 13:02:23.483471 16022 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:02:23.481277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48156 (local address 127.21.235.124:41639)
0809 13:02:23.481484 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.481489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.481827 (+ 338us) server_negotiation.cc:407] Connection header received
0809 13:02:23.481975 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.481980 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.482063 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.482156 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:02:23.483131 (+ 975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48156: BlockingRecv error: recv got EOF from 127.0.0.1:48156 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 13:02:23.484485 16039 catalog_manager.cc:1259] Loaded cluster ID: 233e41cd58c746f9a16243f63b60733f
I20260809 13:02:23.484560 16039 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090: loading cluster ID for follower catalog manager: success
I20260809 13:02:23.487044 16039 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090: acquiring CA information for follower catalog manager: success
I20260809 13:02:23.488765 16039 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5022831b0878469f92478e56911a6090: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 13:02:23.620505 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:23.615126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34866 (local address 127.21.235.125:44027)
0809 13:02:23.615364 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.615371 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.615663 (+ 292us) server_negotiation.cc:407] Connection header received
0809 13:02:23.615901 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.615905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.615992 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.616096 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:23.616543 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.617218 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.618679 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.619184 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.619326 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:23.619767 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:23.619875 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:23.619977 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:23.620141 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 13:02:23.620226 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 13:02:23.632576 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:23.626049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48172 (local address 127.21.235.124:41639)
0809 13:02:23.626287 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.626297 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.626758 (+ 461us) server_negotiation.cc:407] Connection header received
0809 13:02:23.627009 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.627014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.627175 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.627309 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:23.627763 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.628618 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.630095 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.630631 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.630779 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:23.631367 (+ 588us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:23.631917 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:23.632110 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:23.632169 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 13:02:23.632259 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 13:02:23.646385 16221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:23.638444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51742 (local address 127.21.235.126:46173)
0809 13:02:23.640005 (+ 1561us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.640014 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.640040 (+ 26us) server_negotiation.cc:407] Connection header received
0809 13:02:23.640183 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.640191 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.640458 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.640684 (+ 226us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:23.641340 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.642562 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.643875 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.644573 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.644706 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:23.645193 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:23.645666 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:23.645779 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:23.645961 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 13:02:23.646046 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1389,"thread_start_us":1248,"threads_started":1}
I20260809 13:02:23.649536 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:23.638508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34868 (local address 127.21.235.125:44027)
0809 13:02:23.638729 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.638738 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.641469 (+ 2731us) server_negotiation.cc:407] Connection header received
0809 13:02:23.642099 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.642104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.642193 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.642306 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:23.644533 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.645221 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.647275 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.647974 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.648103 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:23.648763 (+ 660us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:23.648899 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:23.648991 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:23.649177 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 13:02:23.649258 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 13:02:23.650386 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:02:23.641676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48186 (local address 127.21.235.124:41639)
0809 13:02:23.641852 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 13:02:23.641857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:02:23.642137 (+ 280us) server_negotiation.cc:407] Connection header received
0809 13:02:23.642310 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:02:23.642315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:02:23.642394 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:02:23.642515 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:02:23.645369 (+ 2854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.646271 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.648114 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:02:23.648738 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:02:23.648852 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:02:23.649576 (+ 724us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:02:23.649707 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:02:23.649798 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 13:02:23.650006 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 13:02:23.650083 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 13:02:23.651073 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 15774
I20260809 13:02:23.652123 15844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:02:23.765770 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 15774
W20260809 13:02:23.791956 15961 connection.cc:535] server connection from 127.0.0.1:48064 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 13:02:23.792443 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 15857
I20260809 13:02:23.793570 15928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:02:23.902364 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 15857
I20260809 13:02:23.926271 22445 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu pid 15943
I20260809 13:02:23.927464 16011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:02:24.036203 22445 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjg7NUQ/build/asan/bin/kudu with pid 15943
2026-08-09T13:02:24Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4814 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9958 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (380275 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST